fix(vk): map the NVN 5-5-5 and BGRA8 formats by their bit layout #924

Open
jcabarca wants to merge 3 commits from jcabarca/Longshot:fix/vk-555-formats into main
Contributor

Type: fix · Risk: low · Tested on: Windows, AMD Radeon RX 9070 XT · Needs: none

Why

Textures in the NVN formats BGR5, BGR5A1, A1BGR5, BGRA8 and BGRA8_SRGB had no Vulkan format, so texture_get() left them out ("NVN format ... has no Vulkan format: its textures are left out") and the game drew without them. RGB5 was mapped to B5G5R5A1, whose bits run the other way, so its colors and alpha came out scrambled.

How to reproduce

Seen on: not seen in the game: found by reading vk_format() in gpu/vk/vk_formats.cpp; any texture in one of these formats is dropped (logged once per format) or shown with the wrong colors
Settings and switches: defaults
Steps: game-free: python -I -B tests/test_555_formats.py --source <main checkout> (the test from this PR); it reads vk_format() and the sampled view's swizzle in vk_texture.cpp
Expected: each of these formats maps to the Vulkan format with its bit layout, and the sampled view fixes R/B order and the missing alpha
Actual: on main five of them map to UNDEFINED and RGB5 to B5G5R5A1; the test fails with ten problems

What changed

vk_format() maps the 5-5-5 and BGRA8 formats by their bit layout, and the sampled view swaps R and B, or reads alpha as one, where the Vulkan format cannot express the NVN one directly.

  • vk: vk_format() in gpu/vk/vk_formats.cpp:
    • BGR5A1 and BGR5 -> A1R5G5B5_UNORM_PACK16; A1BGR5 -> R5G5B5A1_UNORM_PACK16 (new; they were UNDEFINED).
    • RGB5 -> A1R5G5B5_UNORM_PACK16, with RGB5A1 (was B5G5R5A1_UNORM_PACK16).
    • BGRA8 -> B8G8R8A8_UNORM, BGRA8_SRGB -> B8G8R8A8_SRGB (new; they were UNDEFINED).
  • vk: texture_view() in gpu/vk/vk_texture.cpp: BGR5 and RGB5 join the formats whose sampled view reads alpha as one; RGB5A1 and RGB5 get R and B swapped in the view's swizzle.
  • build: V1_NVN in nx2/pipeline_states.py (the VkFormat -> NVN table used to convert old pipeline state lists) follows the mapping: Vulkan format 6 (R5G5B5A1_UNORM_PACK16) now converts to A1BGR5 (0x98), and 7 (B5G5R5A1_UNORM_PACK16) no longer has an NVN twin.
  • tests: tests/test_555_formats.py (new).

Second commit, "leave A1BGR5 textures out where the GPU does not sample R5G5B5A1":

  • vk: vk_format() in gpu/vk/vk_formats.cpp: A1BGR5 maps to R5G5B5A1_UNORM_PACK16 only where r5g5b5a1_sampled() (new, asked once with vkGetPhysicalDeviceFormatProperties) finds VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT in optimalTilingFeatures; elsewhere it is VK_FORMAT_UNDEFINED and the texture is left out, as on main.
  • tests: tests/test_555_formats.py checks that gate; tests/test_pipeline_state_portable.py cuts r5g5b5a1_sampled() into its fixture with vk_format(); tests/test_pipeline_states.py reads a vk_format() case whose return asks such a function.

R5G5B5A1_UNORM_PACK16 has no mandatory support (the mandatory format support tables list A1R5G5B5, not R5G5B5A1). An image made in a format the device does not support for its usage breaks VUID-VkImageCreateInfo-imageCreateMaxMipLevels-02251: "Each of the following values (as described in Image Creation Limits) must not be undefined : imageCreateMaxMipLevels, imageCreateMaxArrayLayers, imageCreateMaxExtent, and imageCreateSampleCounts".

The bug, step by step:

  1. NVN's 16-bit formats name the component in the low bits first: RGB565 maps to B5G6R5_UNORM_PACK16, whose name lists the high bits first.
  2. By the same rule, BGR5A1 (B low, A high) is Vulkan's A1R5G5B5, and A1BGR5 (A low) is R5G5B5A1. vk_format() had no case for them, nor for BGR5, BGRA8 and BGRA8_SRGB: they fell to default: return {VK_FORMAT_UNDEFINED, ...}.
  3. texture_get() returns nullptr for an UNDEFINED format, so those textures were never made or bound.
  4. RGB5 (R low) was mapped to B5G5R5A1, which has R in the high bits and alpha in bit 0: every channel read from the wrong bits.
  5. Vulkan has no 5-5-5 format with R low and alpha high, so RGB5 and RGB5A1 live in A1R5G5B5 (B low) and the sampled view swaps R and B. RGB5 and BGR5 have no alpha bit in the guest data, so their view reads alpha as one.

This breaks no Vulkan rule: the textures were left out or read with the wrong bit layout.

Before (vk_formats.cpp):

    case NVN_FORMAT_RGB5: return {VK_FORMAT_B5G5R5A1_UNORM_PACK16, false, 1, 1};
    ...
    case NVN_FORMAT_RGB5A1: return {VK_FORMAT_A1R5G5B5_UNORM_PACK16, false, 1, 1};
    ...
    case NVN_FORMAT_BGRX8_SRGB: return {VK_FORMAT_B8G8R8A8_SRGB, false, 1, 1};
    default: return {VK_FORMAT_UNDEFINED, false, 1, 1};

After:

    // (the 5-5-5 formats, low bits first as RGB565's: BGR5A1 has B in the low bits, A1BGR5 A)
    case NVN_FORMAT_BGR5A1: return {VK_FORMAT_A1R5G5B5_UNORM_PACK16, false, 1, 1};
    case NVN_FORMAT_BGR5: return {VK_FORMAT_A1R5G5B5_UNORM_PACK16, false, 1, 1};
    case NVN_FORMAT_A1BGR5:
        return {r5g5b5a1_sampled() ? VK_FORMAT_R5G5B5A1_UNORM_PACK16 : VK_FORMAT_UNDEFINED, false, 1, 1};
    ...
    // (RGB5A1 and RGB5 have R in the low bits: the sampled view swaps R and B, and reads RGB5's alpha as one, vk_texture.cpp)
    case NVN_FORMAT_RGB5A1: return {VK_FORMAT_A1R5G5B5_UNORM_PACK16, false, 1, 1};
    case NVN_FORMAT_RGB5: return {VK_FORMAT_A1R5G5B5_UNORM_PACK16, false, 1, 1};
    ...
    case NVN_FORMAT_BGRX8_SRGB: return {VK_FORMAT_B8G8R8A8_SRGB, false, 1, 1};
    case NVN_FORMAT_BGRA8: return {VK_FORMAT_B8G8R8A8_UNORM, false, 1, 1};
    case NVN_FORMAT_BGRA8_SRGB: return {VK_FORMAT_B8G8R8A8_SRGB, false, 1, 1};
    default: return {VK_FORMAT_UNDEFINED, false, 1, 1};

texture_view(), after:

        case NVN_FORMAT_RGBX8_SRGB: case NVN_FORMAT_BGRX8: case NVN_FORMAT_BGRX8_SRGB: case NVN_FORMAT_BGR5:
        case NVN_FORMAT_RGB5:
            for (unsigned& s : sw)
                if (s == NVN_TEXTURE_SWIZZLE_A) s = NVN_TEXTURE_SWIZZLE_ONE;
            break;
        default: break;
        }
        // (RGB5A1 and RGB5 have R in the low bits, their A1R5G5B5 image B)
        if (nf == NVN_FORMAT_RGB5A1 || nf == NVN_FORMAT_RGB5)
            for (unsigned& s : sw)
                s = s == NVN_TEXTURE_SWIZZLE_R ? NVN_TEXTURE_SWIZZLE_B : s == NVN_TEXTURE_SWIZZLE_B ? NVN_TEXTURE_SWIZZLE_R : s;

Measurements

Verdict: it fixes the bug: test_555_formats fails on main and passes here; test_pipeline_state_portable fails on main (its fixture from this PR cuts r5g5b5a1_sampled(), which main lacks) and passes here; test_pipeline_states passes on both (it guards the V1_NVN table). Both interleaved runs gave the same result. No speed change is expected.

Hardware: AMD Radeon RX 9070 XT, Windows; the tests are game-free (they read source or build a cut-out fixture, they do not use the GPU); Python 3.12
Scene: none, game-free
Method: the tests from this PR run against each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gave
How to reproduce: python -I -B tests/<test>.py with --source <checkout> (and --cxx/--headers where the test asks for them) on each side

Before: main @ c167ec67 After: this PR @ 9d050e5d
tests/test_555_formats.py, run 1 fails passes
tests/test_pipeline_state_portable.py, run 1 fails passes
tests/test_pipeline_states.py, run 1 passes passes
tests/test_555_formats.py, run 2 fails passes
tests/test_pipeline_state_portable.py, run 2 fails passes
tests/test_pipeline_states.py, run 2 passes passes
tests/test_555_formats.py, median fails passes
tests/test_pipeline_state_portable.py, median fails passes
tests/test_pipeline_states.py, median passes passes
run 1, main @ c167ec67:
the shipped pipeline state list: 4 pipelines
left out of the shipped pipeline state list: <tmp>\tmp0mrrq_im\assets\pipeline_states\tour.bin.xz: not a pipeline state list this build reads (HKSTATE2, 552-byte records, or HKSTATE1, 528; found HKSTATE9): make it again with this build (docs/development.md, "The pipeline state list")
the shipped pipeline state list: 3 pipelines
pipeline state list: records of both versions, conversion, merge and shipping ok
test_555_formats                     exit 1
test_pipeline_state_portable         exit 1
test_pipeline_states                 exit 0
    main()
    at = fmt.index("case NVN_FORMAT_%s:" % name)
ValueError: substring not found
    main()
    formats = '\n'.join([astc_tables, 'namespace {\n' + block(fmts, 'bool r5g5b5a1_sampled()') + '\n}',
    start = text.index(marker)
ValueError: substring not found
run 1, this PR @ 9d050e5d:
5-5-5, 4-4-4-4 and BGRA8 formats map with their bit layout: ok
PASS: targets mapped as the draw maps them on both GPUs; mesh records the same from native and fallback; v1 and v2 read
the shipped pipeline state list: 4 pipelines
left out of the shipped pipeline state list: <tmp>\tmp7loeb4fn\assets\pipeline_states\tour.bin.xz: not a pipeline state list this build reads (HKSTATE2, 552-byte records, or HKSTATE1, 528; found HKSTATE9): make it again with this build (docs/development.md, "The pipeline state list")
the shipped pipeline state list: 3 pipelines
pipeline state list: records of both versions, conversion, merge and shipping ok
test_555_formats                     exit 0
test_pipeline_state_portable         exit 0
test_pipeline_states                 exit 0
run 2, main @ c167ec67:
the shipped pipeline state list: 4 pipelines
left out of the shipped pipeline state list: <tmp>\tmpf36a6whu\assets\pipeline_states\tour.bin.xz: not a pipeline state list this build reads (HKSTATE2, 552-byte records, or HKSTATE1, 528; found HKSTATE9): make it again with this build (docs/development.md, "The pipeline state list")
the shipped pipeline state list: 3 pipelines
pipeline state list: records of both versions, conversion, merge and shipping ok
test_555_formats                     exit 1
test_pipeline_state_portable         exit 1
test_pipeline_states                 exit 0
    main()
    at = fmt.index("case NVN_FORMAT_%s:" % name)
ValueError: substring not found
    main()
    formats = '\n'.join([astc_tables, 'namespace {\n' + block(fmts, 'bool r5g5b5a1_sampled()') + '\n}',
    start = text.index(marker)
ValueError: substring not found
run 2, this PR @ 9d050e5d:
5-5-5, 4-4-4-4 and BGRA8 formats map with their bit layout: ok
PASS: targets mapped as the draw maps them on both GPUs; mesh records the same from native and fallback; v1 and v2 read
the shipped pipeline state list: 4 pipelines
left out of the shipped pipeline state list: <tmp>\tmpzyf6i89e\assets\pipeline_states\tour.bin.xz: not a pipeline state list this build reads (HKSTATE2, 552-byte records, or HKSTATE1, 528; found HKSTATE9): make it again with this build (docs/development.md, "The pipeline state list")
the shipped pipeline state list: 3 pipelines
pipeline state list: records of both versions, conversion, merge and shipping ok
test_555_formats                     exit 0
test_pipeline_state_portable         exit 0
test_pipeline_states                 exit 0

Flags and settings

None.

Testing

Tested: on this branch, Windows, Python 3.12: test_555_formats passes (fails on main), and the other checks that read vk_formats.cpp or V1_NVN pass: test_pipeline_states, test_pipeline_state_portable, test_native_astc_texture_count. Of the checks that import nx2/pipeline_states.py, these pass: test_dev_packs, test_pipeline_notes_open, test_pipeline_share, test_pipeline_state_computes, test_pipeline_state_counts, test_pipeline_state_sources, test_ship_state_list, test_state_list_plain; test_game_layout, test_lift_cache and test_pipeline_tour fail the same way on main on this PC (a missing program, numpy not installed, a tour run without --vk-profile), unrelated to this change.

Built and played: this commit builds on Windows (llvm-mingw back end) and on Linux (longshot_vk.so, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.

Not tested: a texture in one of these formats looked at in the game: not checked in the game (which textures use them was not traced). Linux was built, not run.

Must be tested: None: A1R5G5B5_UNORM_PACK16 is a format every Vulkan device must sample; B8G8R8A8 must be sampled by every device too; R5G5B5A1_UNORM_PACK16 need not be, and A1BGR5 is left out where vkGetPhysicalDeviceFormatProperties does not report it sampled.

Risks

What could break: textures that were left out now appear; if one of them was wrong for another reason it now shows. RGB5 textures change colors (they were scrambled). Pipeline state lists recorded by older builds with a R5G5B5A1 attachment now convert it to A1BGR5 instead of RGB5A1; B5G5R5A1 ones are no longer converted (no NVN format uses it now).

Turning it off: revert the commit.

Compatibility: Pipeline state lists in version 1 with a 5-5-5 attachment convert as described above; nothing else changes.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched). (no issue fits)
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). (the checks that read the changed code, on this PR, Windows: all pass except test_game_layout, test_lift_cache, test_pipeline_tour, which fail the same way on main on this PC)
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here. (tests/test_555_formats.py)
  • A check under tests/ that reads changed source updated in this PR, not weakened. (tests/test_555_formats.py added; no check removed)
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (built on Windows with llvm-mingw and on Linux, longshot_vk.so without and with AMD FSR 4)
**Type:** fix · **Risk:** low · **Tested on:** Windows, AMD Radeon RX 9070 XT · **Needs:** none ## Why Textures in the NVN formats BGR5, BGR5A1, A1BGR5, BGRA8 and BGRA8_SRGB had no Vulkan format, so `texture_get()` left them out ("NVN format ... has no Vulkan format: its textures are left out") and the game drew without them. RGB5 was mapped to B5G5R5A1, whose bits run the other way, so its colors and alpha came out scrambled. ## How to reproduce **Seen on:** not seen in the game: found by reading `vk_format()` in `gpu/vk/vk_formats.cpp`; any texture in one of these formats is dropped (logged once per format) or shown with the wrong colors **Settings and switches:** defaults **Steps:** game-free: `python -I -B tests/test_555_formats.py --source <main checkout>` (the test from this PR); it reads `vk_format()` and the sampled view's swizzle in `vk_texture.cpp` **Expected:** each of these formats maps to the Vulkan format with its bit layout, and the sampled view fixes R/B order and the missing alpha **Actual:** on main five of them map to UNDEFINED and RGB5 to B5G5R5A1; the test fails with ten problems ## What changed `vk_format()` maps the 5-5-5 and BGRA8 formats by their bit layout, and the sampled view swaps R and B, or reads alpha as one, where the Vulkan format cannot express the NVN one directly. - vk: `vk_format()` in `gpu/vk/vk_formats.cpp`: - BGR5A1 and BGR5 -> `A1R5G5B5_UNORM_PACK16`; A1BGR5 -> `R5G5B5A1_UNORM_PACK16` (new; they were UNDEFINED). - RGB5 -> `A1R5G5B5_UNORM_PACK16`, with RGB5A1 (was `B5G5R5A1_UNORM_PACK16`). - BGRA8 -> `B8G8R8A8_UNORM`, BGRA8_SRGB -> `B8G8R8A8_SRGB` (new; they were UNDEFINED). - vk: `texture_view()` in `gpu/vk/vk_texture.cpp`: BGR5 and RGB5 join the formats whose sampled view reads alpha as one; RGB5A1 and RGB5 get R and B swapped in the view's swizzle. - build: `V1_NVN` in `nx2/pipeline_states.py` (the VkFormat -> NVN table used to convert old pipeline state lists) follows the mapping: Vulkan format 6 (`R5G5B5A1_UNORM_PACK16`) now converts to A1BGR5 (0x98), and 7 (`B5G5R5A1_UNORM_PACK16`) no longer has an NVN twin. - tests: `tests/test_555_formats.py` (new). Second commit, "leave A1BGR5 textures out where the GPU does not sample R5G5B5A1": - vk: `vk_format()` in `gpu/vk/vk_formats.cpp`: A1BGR5 maps to `R5G5B5A1_UNORM_PACK16` only where `r5g5b5a1_sampled()` (new, asked once with `vkGetPhysicalDeviceFormatProperties`) finds `VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT` in `optimalTilingFeatures`; elsewhere it is `VK_FORMAT_UNDEFINED` and the texture is left out, as on main. - tests: `tests/test_555_formats.py` checks that gate; `tests/test_pipeline_state_portable.py` cuts `r5g5b5a1_sampled()` into its fixture with `vk_format()`; `tests/test_pipeline_states.py` reads a `vk_format()` case whose return asks such a function. `R5G5B5A1_UNORM_PACK16` has no mandatory support (the mandatory format support tables list `A1R5G5B5`, not `R5G5B5A1`). An image made in a format the device does not support for its usage breaks VUID-VkImageCreateInfo-imageCreateMaxMipLevels-02251: "Each of the following values (as described in Image Creation Limits) must not be undefined : imageCreateMaxMipLevels, imageCreateMaxArrayLayers, imageCreateMaxExtent, and imageCreateSampleCounts". The bug, step by step: 1. NVN's 16-bit formats name the component in the low bits first: RGB565 maps to `B5G6R5_UNORM_PACK16`, whose name lists the high bits first. 2. By the same rule, BGR5A1 (B low, A high) is Vulkan's `A1R5G5B5`, and A1BGR5 (A low) is `R5G5B5A1`. `vk_format()` had no case for them, nor for BGR5, BGRA8 and BGRA8_SRGB: they fell to `default: return {VK_FORMAT_UNDEFINED, ...}`. 3. `texture_get()` returns `nullptr` for an UNDEFINED format, so those textures were never made or bound. 4. RGB5 (R low) was mapped to `B5G5R5A1`, which has R in the high bits and alpha in bit 0: every channel read from the wrong bits. 5. Vulkan has no 5-5-5 format with R low and alpha high, so RGB5 and RGB5A1 live in `A1R5G5B5` (B low) and the sampled view swaps R and B. RGB5 and BGR5 have no alpha bit in the guest data, so their view reads alpha as one. This breaks no Vulkan rule: the textures were left out or read with the wrong bit layout. Before (`vk_formats.cpp`): ```cpp case NVN_FORMAT_RGB5: return {VK_FORMAT_B5G5R5A1_UNORM_PACK16, false, 1, 1}; ... case NVN_FORMAT_RGB5A1: return {VK_FORMAT_A1R5G5B5_UNORM_PACK16, false, 1, 1}; ... case NVN_FORMAT_BGRX8_SRGB: return {VK_FORMAT_B8G8R8A8_SRGB, false, 1, 1}; default: return {VK_FORMAT_UNDEFINED, false, 1, 1}; ``` After: ```cpp // (the 5-5-5 formats, low bits first as RGB565's: BGR5A1 has B in the low bits, A1BGR5 A) case NVN_FORMAT_BGR5A1: return {VK_FORMAT_A1R5G5B5_UNORM_PACK16, false, 1, 1}; case NVN_FORMAT_BGR5: return {VK_FORMAT_A1R5G5B5_UNORM_PACK16, false, 1, 1}; case NVN_FORMAT_A1BGR5: return {r5g5b5a1_sampled() ? VK_FORMAT_R5G5B5A1_UNORM_PACK16 : VK_FORMAT_UNDEFINED, false, 1, 1}; ... // (RGB5A1 and RGB5 have R in the low bits: the sampled view swaps R and B, and reads RGB5's alpha as one, vk_texture.cpp) case NVN_FORMAT_RGB5A1: return {VK_FORMAT_A1R5G5B5_UNORM_PACK16, false, 1, 1}; case NVN_FORMAT_RGB5: return {VK_FORMAT_A1R5G5B5_UNORM_PACK16, false, 1, 1}; ... case NVN_FORMAT_BGRX8_SRGB: return {VK_FORMAT_B8G8R8A8_SRGB, false, 1, 1}; case NVN_FORMAT_BGRA8: return {VK_FORMAT_B8G8R8A8_UNORM, false, 1, 1}; case NVN_FORMAT_BGRA8_SRGB: return {VK_FORMAT_B8G8R8A8_SRGB, false, 1, 1}; default: return {VK_FORMAT_UNDEFINED, false, 1, 1}; ``` `texture_view()`, after: ```cpp case NVN_FORMAT_RGBX8_SRGB: case NVN_FORMAT_BGRX8: case NVN_FORMAT_BGRX8_SRGB: case NVN_FORMAT_BGR5: case NVN_FORMAT_RGB5: for (unsigned& s : sw) if (s == NVN_TEXTURE_SWIZZLE_A) s = NVN_TEXTURE_SWIZZLE_ONE; break; default: break; } // (RGB5A1 and RGB5 have R in the low bits, their A1R5G5B5 image B) if (nf == NVN_FORMAT_RGB5A1 || nf == NVN_FORMAT_RGB5) for (unsigned& s : sw) s = s == NVN_TEXTURE_SWIZZLE_R ? NVN_TEXTURE_SWIZZLE_B : s == NVN_TEXTURE_SWIZZLE_B ? NVN_TEXTURE_SWIZZLE_R : s; ``` ## Measurements **Verdict:** it fixes the bug: `test_555_formats` fails on main and passes here; `test_pipeline_state_portable` fails on main (its fixture from this PR cuts `r5g5b5a1_sampled()`, which main lacks) and passes here; `test_pipeline_states` passes on both (it guards the V1_NVN table). Both interleaved runs gave the same result. No speed change is expected. Hardware: AMD Radeon RX 9070 XT, Windows; the tests are game-free (they read source or build a cut-out fixture, they do not use the GPU); Python 3.12 Scene: none, game-free Method: the tests from this PR run against each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gave How to reproduce: `python -I -B tests/<test>.py` with `--source <checkout>` (and `--cxx`/`--headers` where the test asks for them) on each side | | Before: main @ c167ec67 | After: this PR @ 9d050e5d | |---|---|---| | `tests/test_555_formats.py`, run 1 | fails | passes | | `tests/test_pipeline_state_portable.py`, run 1 | fails | passes | | `tests/test_pipeline_states.py`, run 1 | passes | passes | | `tests/test_555_formats.py`, run 2 | fails | passes | | `tests/test_pipeline_state_portable.py`, run 2 | fails | passes | | `tests/test_pipeline_states.py`, run 2 | passes | passes | | `tests/test_555_formats.py`, median | fails | passes | | `tests/test_pipeline_state_portable.py`, median | fails | passes | | `tests/test_pipeline_states.py`, median | passes | passes | ``` run 1, main @ c167ec67: the shipped pipeline state list: 4 pipelines left out of the shipped pipeline state list: <tmp>\tmp0mrrq_im\assets\pipeline_states\tour.bin.xz: not a pipeline state list this build reads (HKSTATE2, 552-byte records, or HKSTATE1, 528; found HKSTATE9): make it again with this build (docs/development.md, "The pipeline state list") the shipped pipeline state list: 3 pipelines pipeline state list: records of both versions, conversion, merge and shipping ok test_555_formats exit 1 test_pipeline_state_portable exit 1 test_pipeline_states exit 0 main() at = fmt.index("case NVN_FORMAT_%s:" % name) ValueError: substring not found main() formats = '\n'.join([astc_tables, 'namespace {\n' + block(fmts, 'bool r5g5b5a1_sampled()') + '\n}', start = text.index(marker) ValueError: substring not found run 1, this PR @ 9d050e5d: 5-5-5, 4-4-4-4 and BGRA8 formats map with their bit layout: ok PASS: targets mapped as the draw maps them on both GPUs; mesh records the same from native and fallback; v1 and v2 read the shipped pipeline state list: 4 pipelines left out of the shipped pipeline state list: <tmp>\tmp7loeb4fn\assets\pipeline_states\tour.bin.xz: not a pipeline state list this build reads (HKSTATE2, 552-byte records, or HKSTATE1, 528; found HKSTATE9): make it again with this build (docs/development.md, "The pipeline state list") the shipped pipeline state list: 3 pipelines pipeline state list: records of both versions, conversion, merge and shipping ok test_555_formats exit 0 test_pipeline_state_portable exit 0 test_pipeline_states exit 0 run 2, main @ c167ec67: the shipped pipeline state list: 4 pipelines left out of the shipped pipeline state list: <tmp>\tmpf36a6whu\assets\pipeline_states\tour.bin.xz: not a pipeline state list this build reads (HKSTATE2, 552-byte records, or HKSTATE1, 528; found HKSTATE9): make it again with this build (docs/development.md, "The pipeline state list") the shipped pipeline state list: 3 pipelines pipeline state list: records of both versions, conversion, merge and shipping ok test_555_formats exit 1 test_pipeline_state_portable exit 1 test_pipeline_states exit 0 main() at = fmt.index("case NVN_FORMAT_%s:" % name) ValueError: substring not found main() formats = '\n'.join([astc_tables, 'namespace {\n' + block(fmts, 'bool r5g5b5a1_sampled()') + '\n}', start = text.index(marker) ValueError: substring not found run 2, this PR @ 9d050e5d: 5-5-5, 4-4-4-4 and BGRA8 formats map with their bit layout: ok PASS: targets mapped as the draw maps them on both GPUs; mesh records the same from native and fallback; v1 and v2 read the shipped pipeline state list: 4 pipelines left out of the shipped pipeline state list: <tmp>\tmpzyf6i89e\assets\pipeline_states\tour.bin.xz: not a pipeline state list this build reads (HKSTATE2, 552-byte records, or HKSTATE1, 528; found HKSTATE9): make it again with this build (docs/development.md, "The pipeline state list") the shipped pipeline state list: 3 pipelines pipeline state list: records of both versions, conversion, merge and shipping ok test_555_formats exit 0 test_pipeline_state_portable exit 0 test_pipeline_states exit 0 ``` ### Flags and settings None. ## Testing **Tested:** on this branch, Windows, Python 3.12: `test_555_formats` passes (fails on main), and the other checks that read `vk_formats.cpp` or `V1_NVN` pass: `test_pipeline_states`, `test_pipeline_state_portable`, `test_native_astc_texture_count`. Of the checks that import `nx2/pipeline_states.py`, these pass: `test_dev_packs`, `test_pipeline_notes_open`, `test_pipeline_share`, `test_pipeline_state_computes`, `test_pipeline_state_counts`, `test_pipeline_state_sources`, `test_ship_state_list`, `test_state_list_plain`; `test_game_layout`, `test_lift_cache` and `test_pipeline_tour` fail the same way on main on this PC (a missing program, numpy not installed, a tour run without `--vk-profile`), unrelated to this change. **Built and played:** this commit builds on Windows (llvm-mingw back end) and on Linux (`longshot_vk.so`, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally. **Not tested:** a texture in one of these formats looked at in the game: not checked in the game (which textures use them was not traced). Linux was built, not run. **Must be tested:** None: `A1R5G5B5_UNORM_PACK16` is a format every Vulkan device must sample; `B8G8R8A8` must be sampled by every device too; `R5G5B5A1_UNORM_PACK16` need not be, and A1BGR5 is left out where `vkGetPhysicalDeviceFormatProperties` does not report it sampled. ## Risks **What could break:** textures that were left out now appear; if one of them was wrong for another reason it now shows. RGB5 textures change colors (they were scrambled). Pipeline state lists recorded by older builds with a `R5G5B5A1` attachment now convert it to A1BGR5 instead of RGB5A1; `B5G5R5A1` ones are no longer converted (no NVN format uses it now). **Turning it off:** revert the commit. **Compatibility:** Pipeline state lists in version 1 with a 5-5-5 attachment convert as described above; nothing else changes. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). (no issue fits) - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). (the checks that read the changed code, on this PR, Windows: all pass except `test_game_layout`, `test_lift_cache`, `test_pipeline_tour`, which fail the same way on `main` on this PC) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. (`tests/test_555_formats.py`) - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. (`tests/test_555_formats.py` added; no check removed) - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (built on Windows with llvm-mingw and on Linux, `longshot_vk.so` without and with AMD FSR 4)
fix(vk): map the NVN 5-5-5 and BGRA8 formats by their bit layout
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
034b3aaf4a
vk_format left BGR5, BGR5A1, A1BGR5, BGRA8 and BGRA8_SRGB UNDEFINED and mapped RGB5 to B5G5R5A1, whose bits run the other way. BGR5A1/BGR5 now map to A1R5G5B5, A1BGR5 to R5G5B5A1, RGB5 joins RGB5A1 in A1R5G5B5 with the sampled view swapping R and B, BGR5 and RGB5 read alpha as one, and BGRA8/BGRA8_SRGB map to B8G8R8A8. V1_NVN (nx2/pipeline_states.py) follows: R5G5B5A1 converts to A1BGR5, B5G5R5A1 no longer has an NVN twin. tests/test_555_formats.py checks the mappings and the view swizzle.

Assisted-by: GitHub Copilot
jcabarca changed title from WIP: fix(vk): map the NVN 5-5-5 and BGRA8 formats by their bit layout to fix(vk): map the NVN 5-5-5 and BGRA8 formats by their bit layout 2026-11-15 20:49:46 +00:00
Member

Code-only review of 034b3aaf4a; informational findings, with final approval left to another reviewer.

No new correctness defect found in the mapping and sampled-view changes. RGB5/RGB5A1 remap the swizzle sources after descriptor/view selection, and the alpha-one substitution composes with that remap. The version-1 pipeline conversion change removes the obsolete B5G5R5A1 mapping and adds the newly supported R5G5B5A1 mapping consistently with the format switch.

The reported Windows RX 9070 XT game run and Windows/Linux builds are useful smoke coverage. The description correctly notes that no texture using the affected formats was identified and visually checked; those results therefore do not establish the advertised color correction in-game. A targeted packed-color sample/readback or before/after scene using the affected formats would close that gap. The new test_555_formats.py mostly matches source spelling: for example, removing the alpha assignment while leaving the case labels would still satisfy its alpha-one check. Prefer a behavioral packed-pixel/swizzle fixture, including view overrides and BGR5 alpha, alongside the mapping checks.

Merge coordination: PR 452 implements the same RGB5/RGB5A1 production correction with an exhaustive CPU fixture; this PR additionally covers BGR5/BGR5A1/A1BGR5 and BGRA8. Consolidate the shared fix once and preserve the stronger coverage. If both tests remain, reconcile this test's exact // (X formats read alpha as one) marker with 452's changed comment, or the combined test will fail before checking the mapping. The broader PR 451 also overlaps these format mappings.

Read the complete diff, surrounding sampled-view/cache code, NVN enums, pipeline conversion and its existing checks, and the overlapping 452 production diff. No tests, builds, generators or game runs performed. No active-work claim was present in the checked PR thread/team snapshot; the current head has no previous formal reviews.

Code-only review of 034b3aaf4a92ec69a39c5e27a650e4928f61d549; informational findings, with final approval left to another reviewer. No new correctness defect found in the mapping and sampled-view changes. RGB5/RGB5A1 remap the swizzle sources after descriptor/view selection, and the alpha-one substitution composes with that remap. The version-1 pipeline conversion change removes the obsolete B5G5R5A1 mapping and adds the newly supported R5G5B5A1 mapping consistently with the format switch. The reported Windows RX 9070 XT game run and Windows/Linux builds are useful smoke coverage. The description correctly notes that no texture using the affected formats was identified and visually checked; those results therefore do not establish the advertised color correction in-game. A targeted packed-color sample/readback or before/after scene using the affected formats would close that gap. The new `test_555_formats.py` mostly matches source spelling: for example, removing the alpha assignment while leaving the case labels would still satisfy its alpha-one check. Prefer a behavioral packed-pixel/swizzle fixture, including view overrides and BGR5 alpha, alongside the mapping checks. Merge coordination: [PR 452](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/452) implements the same RGB5/RGB5A1 production correction with an exhaustive CPU fixture; this PR additionally covers BGR5/BGR5A1/A1BGR5 and BGRA8. Consolidate the shared fix once and preserve the stronger coverage. If both tests remain, reconcile this test's exact `// (X formats read alpha as one)` marker with 452's changed comment, or the combined test will fail before checking the mapping. The broader [PR 451](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/451) also overlaps these format mappings. Read the complete diff, surrounding sampled-view/cache code, NVN enums, pipeline conversion and its existing checks, and the overlapping 452 production diff. No tests, builds, generators or game runs performed. No active-work claim was present in the checked PR thread/team snapshot; the current head has no previous formal reviews.
jcabarca force-pushed fix/vk-555-formats from 034b3aaf4a
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to 9d050e5ddf
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-16 10:30:24 +00:00
Compare
Author
Contributor

New commit, "leave A1BGR5 textures out where the GPU does not sample R5G5B5A1" (9d050e5d); the branch is rebased onto main @ c167ec67.

R5G5B5A1_UNORM_PACK16 has no mandatory support (the mandatory format support tables list A1R5G5B5, not R5G5B5A1). An image made in a format the device does not support for its usage breaks VUID-VkImageCreateInfo-imageCreateMaxMipLevels-02251: "Each of the following values (as described in Image Creation Limits) must not be undefined : imageCreateMaxMipLevels, imageCreateMaxArrayLayers, imageCreateMaxExtent, and imageCreateSampleCounts".

  • vk: vk_format() in gpu/vk/vk_formats.cpp: A1BGR5 maps to R5G5B5A1_UNORM_PACK16 only where r5g5b5a1_sampled() (new, asked once with vkGetPhysicalDeviceFormatProperties) finds VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT in optimalTilingFeatures; elsewhere it is VK_FORMAT_UNDEFINED and the texture is left out, as on main.
  • tests: tests/test_555_formats.py checks that gate; tests/test_pipeline_state_portable.py cuts r5g5b5a1_sampled() into its fixture with vk_format(); tests/test_pipeline_states.py reads a vk_format() case whose return asks such a function.

test_555_formats fails on main and passes here; test_pipeline_state_portable fails on main (its fixture from this PR cuts r5g5b5a1_sampled(), which main lacks) and passes here; test_pipeline_states passes on both (it guards the V1_NVN table). Run twice each side, interleaved; What changed and Measurements in the description are updated.

New commit, "leave A1BGR5 textures out where the GPU does not sample R5G5B5A1" (9d050e5d); the branch is rebased onto main @ c167ec67. `R5G5B5A1_UNORM_PACK16` has no mandatory support (the mandatory format support tables list `A1R5G5B5`, not `R5G5B5A1`). An image made in a format the device does not support for its usage breaks VUID-VkImageCreateInfo-imageCreateMaxMipLevels-02251: "Each of the following values (as described in Image Creation Limits) must not be undefined : imageCreateMaxMipLevels, imageCreateMaxArrayLayers, imageCreateMaxExtent, and imageCreateSampleCounts". - vk: `vk_format()` in `gpu/vk/vk_formats.cpp`: A1BGR5 maps to `R5G5B5A1_UNORM_PACK16` only where `r5g5b5a1_sampled()` (new, asked once with `vkGetPhysicalDeviceFormatProperties`) finds `VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT` in `optimalTilingFeatures`; elsewhere it is `VK_FORMAT_UNDEFINED` and the texture is left out, as on main. - tests: `tests/test_555_formats.py` checks that gate; `tests/test_pipeline_state_portable.py` cuts `r5g5b5a1_sampled()` into its fixture with `vk_format()`; `tests/test_pipeline_states.py` reads a `vk_format()` case whose return asks such a function. `test_555_formats` fails on main and passes here; `test_pipeline_state_portable` fails on main (its fixture from this PR cuts `r5g5b5a1_sampled()`, which main lacks) and passes here; `test_pipeline_states` passes on both (it guards the V1_NVN table). Run twice each side, interleaved; What changed and Measurements in the description are updated.
Author
Contributor

This commit (both commits of the PR) builds on Windows (llvm-mingw back end) and on Linux (longshot_vk.so, without and with AMD FSR 4), and was played on Windows, AMD Radeon RX 9070 XT: the game runs normally. The description's Built and played line and the Host code builds box are updated.

This commit (both commits of the PR) builds on Windows (llvm-mingw back end) and on Linux (`longshot_vk.so`, without and with AMD FSR 4), and was played on Windows, AMD Radeon RX 9070 XT: the game runs normally. The description's Built and played line and the Host code builds box are updated.
Merge branch 'main' into fix/vk-555-formats
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 5s
2d3281c0c0
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 5s
This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/vk-555-formats:jcabarca-fix/vk-555-formats'
git switch 'jcabarca-fix/vk-555-formats'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!924
No description provided.