perf(vk): keep big ASTC textures as BC7 (RX_VK_ASTC_BC7) #2226

github-sync · 2026-11-11T09:28:13Z

DFN-master commented

2026-11-11 09:28:13 +00:00

(Migrated from github.com)

Big decoded ASTC textures (terrain, walls, the material arrays) are re-encoded to BC3 today to keep them at 8 bits a texel. BC7 mode 6 uses the same 8 bits with much better gradients and alpha, so the grass, paths, stone and tree textures lose visibly less.

RX_VK_ASTC_BC7=1 keeps those textures as BC7. The encoder is a compute shader port of the WebGPU back end's CPU encoder (gpu/render/rx_bc7.cpp): a covariance fit, 7-bit endpoints with p-bits and 4-bit indices, 16 bytes a block, layer-major as the BC3 path writes. Off unless set; the BC3 path is unchanged, and RX_VK_ASTC_RGBA8=1 still wins when both are set.

Tested at 1920x1080 on an RTX 3080: Kokiri Forest holds 60 fps, the same graphics memory as BC3, and no texture artifacts. The shader source and the generated header are checked in; scripts/make_bc7_shader.py rebuilds the header with glslang.

Big decoded ASTC textures (terrain, walls, the material arrays) are re-encoded to BC3 today to keep them at 8 bits a texel. BC7 mode 6 uses the same 8 bits with much better gradients and alpha, so the grass, paths, stone and tree textures lose visibly less. `RX_VK_ASTC_BC7=1` keeps those textures as BC7. The encoder is a compute shader port of the WebGPU back end's CPU encoder (`gpu/render/rx_bc7.cpp`): a covariance fit, 7-bit endpoints with p-bits and 4-bit indices, 16 bytes a block, layer-major as the BC3 path writes. Off unless set; the BC3 path is unchanged, and `RX_VK_ASTC_RGBA8=1` still wins when both are set. Tested at 1920x1080 on an RTX 3080: Kokiri Forest holds 60 fps, the same graphics memory as BC3, and no texture artifacts. The shader source and the generated header are checked in; `scripts/make_bc7_shader.py` rebuilds the header with glslang.

hckrman101 (Migrated from github.com) reviewed

2026-11-11 17:37:08 +00:00

hckrman101 (Migrated from github.com) left a comment

Thanks, this is a nice opt-in: off unless RX_VK_ASTC_BC7=1, the BC3 path untouched, and RX_VK_ASTC_RGBA8 still wins.

What I checked and looks right:

Mode 6 packing: mode bits, R0 R1 G0 G1 B0 B1 A0 A1 at 7 bits, the two p-bits, a 3-bit anchor index then 15 4-bit ones (128 bits). The anchor swap is sound because the weight table is symmetric (w[15-k] = 64 - w[k]).
sRGB vs UNORM: the image is BC7_UNORM and view_format_of picks BC7_SRGB/UNORM through linear_twin exactly as BC3 did. The scratch holds the stored bytes, so sRGB data stays sRGB.
Layout: bc_encode_into always pushes bc3 = 1, out_offset = 0, so the shader ignoring bc3 is fine, and 16 bytes a block matches the copy.
Storage/attachment views of a bc texture are still refused, partial uploads and CPU writes still go through the tex.bc checks, and the AMD resized path goes through bc_encode_into too. HDR ASTC is unchanged (both paths encode the decoder's RGBA8).
Budget: create counts any x.bc image at 1 byte a texel, so BC7 costs the same as BC3, including after #2288.
GPU support: Vulkan 1.3 is required, so the SPIR-V 1.5 header is fine, and BC7 sampling is universal on desktop.

What I'd like before merge:

A test. There's no check in tests/. The repo's convention (see tests/test_fg_scale_sharpen.py) is that the shader's test holds the glslang command and checks that the _spv.h was built from this exact .comp (a hash on its first line). With a standalone script, the header can drift from the source without anyone noticing. A game-free check of the packing would also be cheap, e.g. a small CPU decode of a mode-6 block cut from the shader's logic, or a comparison against gpu/render/rx_bc7.cpp. Please register it with run_all.py the usual way.
Screenshots (CONTRIBUTING.md asks for them when a change shows on screen): BC3 vs BC7 of the same spot. Please include alpha-tested foliage (grass, tree leaves) as well as the ground. Mode 6 fits RGB and alpha on one shared line, while BC3 encodes alpha in its own block. So for cutout textures, where alpha doesn't follow colour, BC7 mode 6 can do worse at the alpha-test edge than BC3, and "much better alpha" may not hold everywhere. If it shows, mode 5 (separate alpha) is the usual fix.
Encode cost off NVIDIA. It was only tested on a 3080. Each invocation keeps px[16][4], a 16x4 palette and two Fits with 16 indices, which is heavy on registers and may spill on AMD and Intel iGPUs. The 90-layer material arrays are encoded one layer per run. If anyone can, vk: slow upload log lines for BC3 vs BC7 on an AMD or Intel card would show whether loading areas stutter more.

CI: the Windows build passes. The AppImage failure is the same apt network timeout main has right now (not this PR). The Python source checks have not run yet (pending).

Thanks, this is a nice opt-in: off unless `RX_VK_ASTC_BC7=1`, the BC3 path untouched, and `RX_VK_ASTC_RGBA8` still wins. What I checked and looks right: - Mode 6 packing: mode bits, R0 R1 G0 G1 B0 B1 A0 A1 at 7 bits, the two p-bits, a 3-bit anchor index then 15 4-bit ones (128 bits). The anchor swap is sound because the weight table is symmetric (w[15-k] = 64 - w[k]). - sRGB vs UNORM: the image is BC7_UNORM and `view_format_of` picks BC7_SRGB/UNORM through `linear_twin` exactly as BC3 did. The scratch holds the stored bytes, so sRGB data stays sRGB. - Layout: `bc_encode_into` always pushes `bc3 = 1, out_offset = 0`, so the shader ignoring `bc3` is fine, and 16 bytes a block matches the copy. - Storage/attachment views of a `bc` texture are still refused, partial uploads and CPU writes still go through the `tex.bc` checks, and the AMD resized path goes through `bc_encode_into` too. HDR ASTC is unchanged (both paths encode the decoder's RGBA8). - Budget: `create` counts any `x.bc` image at 1 byte a texel, so BC7 costs the same as BC3, including after #2288. - GPU support: Vulkan 1.3 is required, so the SPIR-V 1.5 header is fine, and BC7 sampling is universal on desktop. What I'd like before merge: 1. **A test.** There's no check in `tests/`. The repo's convention (see `tests/test_fg_scale_sharpen.py`) is that the shader's test holds the glslang command and checks that the `_spv.h` was built from this exact `.comp` (a hash on its first line). With a standalone script, the header can drift from the source without anyone noticing. A game-free check of the packing would also be cheap, e.g. a small CPU decode of a mode-6 block cut from the shader's logic, or a comparison against `gpu/render/rx_bc7.cpp`. Please register it with run_all.py the usual way. 2. **Screenshots** (CONTRIBUTING.md asks for them when a change shows on screen): BC3 vs BC7 of the same spot. Please include alpha-tested foliage (grass, tree leaves) as well as the ground. Mode 6 fits RGB and alpha on one shared line, while BC3 encodes alpha in its own block. So for cutout textures, where alpha doesn't follow colour, BC7 mode 6 can do worse at the alpha-test edge than BC3, and "much better alpha" may not hold everywhere. If it shows, mode 5 (separate alpha) is the usual fix. 3. **Encode cost off NVIDIA.** It was only tested on a 3080. Each invocation keeps `px[16][4]`, a 16x4 palette and two `Fit`s with 16 indices, which is heavy on registers and may spill on AMD and Intel iGPUs. The 90-layer material arrays are encoded one layer per run. If anyone can, `vk: slow upload` log lines for BC3 vs BC7 on an AMD or Intel card would show whether loading areas stutter more. CI: the Windows build passes. The AppImage failure is the same apt network timeout main has right now (not this PR). The Python source checks have not run yet (pending).

gpu/vk/host_shaders/bc7_encoder.comp

					
				@ -0,0 +12,4 @@

				// encoder of the WebGPU back end (gpu/render/rx_bc7.cpp, RX_ASTC_BC7) as a compute shader: BC7's

				// mode 6 keeps gradients and alpha far better than BC3 at the same 8 bits a texel.

				//

				// Binding 0: the RGBA8 scratch (one layer per dispatch z).

Nit: the other host shaders and their _spv.h headers start with SPDX lines and a provenance note ("NOT a Nintendo file ... compiled from ..."). It would be good to match that here and in bc7_encoder_comp_spv.h.

Nit: the other host shaders and their `_spv.h` headers start with SPDX lines and a provenance note ("NOT a Nintendo file ... compiled from ..."). It would be good to match that here and in `bc7_encoder_comp_spv.h`.

gpu/vk/host_shaders/bc7_encoder.comp

					
				@ -0,0 +140,4 @@

				        for (int a = 0; a < 4; ++a)

				            for (int b = 0; b < 4; ++b) cov[a][b] += d[a] * d[b];

				    }

				    float axis[4] = {cov[0][0], cov[1][1], cov[2][2], cov[3][3]};

Nit (inherited from rx_bc7.cpp): starting the power iteration from the diagonal can land exactly orthogonal to the principal axis. One example is R and G anticorrelated with equal variance (a red-to-green ramp), B and A constant: n comes out 0, the loop breaks, every texel projects to 0, and the block becomes flat at the mean. The refit can't rescue it because every index is the same, so det = 0. Restarting from the covariance row with the largest norm when the first step collapses would avoid it.

Nit (inherited from rx_bc7.cpp): starting the power iteration from the diagonal can land exactly orthogonal to the principal axis. One example is R and G anticorrelated with equal variance (a red-to-green ramp), B and A constant: `n` comes out 0, the loop breaks, every texel projects to 0, and the block becomes flat at the mean. The refit can't rescue it because every index is the same, so det = 0. Restarting from the covariance row with the largest norm when the first step collapses would avoid it.

gpu/vk/vk_texture.cpp

					
				@ -712,3 +744,3 @@

				    state = 1;

				    logf("vk: big ASTC textures are kept as BC3 (re-encoded on the GPU; ASTC as RGBA8 or RX_VK_ASTC_RGBA8=1: as RGBA8)");

				    logf("vk: big ASTC textures are kept as %s (re-encoded on the GPU; ASTC as RGBA8 or RX_VK_ASTC_RGBA8=1: as RGBA8)", bc7 ? "BC7" : "BC3");

				    return true;

If BC7 isn't sampleable here, or the BC7 pipeline fails at line 731, bc_ready returns false and every big ASTC texture becomes RGBA8, which is 4x the memory. Falling back to BC3 (clear a flag so bc7_enabled() reads false afterwards) would be safer. It's unlikely on desktop, so this is a nit.

If BC7 isn't sampleable here, or the BC7 pipeline fails at line 731, `bc_ready` returns false and every big ASTC texture becomes RGBA8, which is 4x the memory. Falling back to BC3 (clear a flag so `bc7_enabled()` reads false afterwards) would be safer. It's unlikely on desktop, so this is a nit.

gpu/vk/vk_texture.cpp

Nit: stray blank line.

Joey-Magnus (Migrated from github.com) reviewed

2026-11-11 21:23:35 +00:00

Joey-Magnus (Migrated from github.com) left a comment

Opt-in, so the risk is low, but a few things:

The new shader isn't in the shader-header drift check. Main builds headers from the SHADERS list in scripts/host_shaders.py, which tests/test_host_shaders.py checks. This PR adds its own scripts/make_bc7_shader.py with a different target (vulkan1.2), so bc7_encoder_comp_spv.h can drift without anything failing. Please add bc7_encoder to that list and drop the private script.
If the BC7 pipeline fails to build (vk_texture.cpp:731), bc_ready() returns false and turns off BC3 too, so big ASTC textures become RGBA8 (about 4x the memory) instead of falling back to BC3.
The earlier review asks are still open: a packing test, BC3 vs BC7 screenshots, and a check on alpha-tested foliage (mode 6 fits colour and alpha on one line).
The test_native_astc_texture_count.py edit duplicates #2397 and can be dropped.

Opt-in, so the risk is low, but a few things: - The new shader isn't in the shader-header drift check. Main builds headers from the `SHADERS` list in `scripts/host_shaders.py`, which `tests/test_host_shaders.py` checks. This PR adds its own `scripts/make_bc7_shader.py` with a different target (`vulkan1.2`), so `bc7_encoder_comp_spv.h` can drift without anything failing. Please add `bc7_encoder` to that list and drop the private script. - If the BC7 pipeline fails to build (`vk_texture.cpp:731`), `bc_ready()` returns false and turns off BC3 too, so big ASTC textures become RGBA8 (about 4x the memory) instead of falling back to BC3. - The earlier review asks are still open: a packing test, BC3 vs BC7 screenshots, and a check on alpha-tested foliage (mode 6 fits colour and alpha on one line). - The `test_native_astc_texture_count.py` edit duplicates #2397 and can be dropped.

hckrman101 (Migrated from github.com) reviewed

2026-11-11 22:21:04 +00:00

hckrman101 (Migrated from github.com) left a comment

Re-checked after the push. It's a rebase onto main: the PR's diff is byte-for-byte the same as the commit I reviewed, so nothing from the first round has changed yet. CI is all green now (the Python source checks were still pending last time) and it merges cleanly.

Still open:

The header can drift from its source. I agree with @Joey-Magnus here, and that's the simplest way to cover the first half of my point 1: add bc7_encoder to SHADERS in scripts/host_shaders.py and drop scripts/make_bc7_shader.py. tests/test_host_shaders.py then recompiles it and fails on a mismatch, and the header gets the same opening comment as the others. Note that host_shaders.py builds with --target-env vulkan1.1, not the vulkan1.2 the private script uses, so the header has to be regenerated that way.
A packing check (a CPU decode of a mode-6 block, or a comparison with gpu/render/rx_bc7.cpp) would still be nice to have, but with item 1 done I wouldn't hold the PR for it.
Screenshots: BC3 vs BC7 at the same spot, including alpha-tested foliage (mode 6 fits colour and alpha on one line).
The fallback when the BC7 pipeline fails (inline thread on vk_texture.cpp): the textures become RGBA8 rather than BC3. Joey raised it too. It's unlikely on desktop, but it's a one-flag fix.

The nits (the power-iteration restart, the SPDX/provenance lines, the stray blank line) are also unchanged and still optional. Joey's note about the test_native_astc_texture_count.py edit doesn't apply: the PR's diff doesn't touch that file (main already has #2397).

Re-checked after the push. It's a rebase onto main: the PR's diff is byte-for-byte the same as the commit I reviewed, so nothing from the first round has changed yet. CI is all green now (the Python source checks were still pending last time) and it merges cleanly. Still open: 1. **The header can drift from its source.** I agree with @Joey-Magnus here, and that's the simplest way to cover the first half of my point 1: add `bc7_encoder` to `SHADERS` in `scripts/host_shaders.py` and drop `scripts/make_bc7_shader.py`. `tests/test_host_shaders.py` then recompiles it and fails on a mismatch, and the header gets the same opening comment as the others. Note that `host_shaders.py` builds with `--target-env vulkan1.1`, not the `vulkan1.2` the private script uses, so the header has to be regenerated that way. 2. **A packing check** (a CPU decode of a mode-6 block, or a comparison with `gpu/render/rx_bc7.cpp`) would still be nice to have, but with item 1 done I wouldn't hold the PR for it. 3. **Screenshots**: BC3 vs BC7 at the same spot, including alpha-tested foliage (mode 6 fits colour and alpha on one line). 4. **The fallback when the BC7 pipeline fails** (inline thread on `vk_texture.cpp`): the textures become RGBA8 rather than BC3. Joey raised it too. It's unlikely on desktop, but it's a one-flag fix. The nits (the power-iteration restart, the SPDX/provenance lines, the stray blank line) are also unchanged and still optional. Joey's note about the `test_native_astc_texture_count.py` edit doesn't apply: the PR's diff doesn't touch that file (main already has #2397).

DFN-master commented

2026-11-11 22:42:19 +00:00

(Migrated from github.com)

Thanks — the new head (57e53f8b, rebased on a5b955fb) addresses the code points:

Header drift: bc7_encoder is now in scripts/host_shaders.py's SHADERS; the private scripts/make_bc7_shader.py is gone; the header was regenerated with glslang 16.6.0 at vulkan1.1, and tests/test_host_shaders.py passes. The shader and header have the SPDX/provenance opening lines.
Fallback: if BC7 is unsupported or its pipeline cannot be made, bc7_ok turns the encoder and the textures to BC3 from then on — not RGBA8.
Power iteration: if the first step from the diagonal collapses, it restarts once from the covariance row with the largest norm (the R/G anticorrelated, B/A constant case), then the usual iteration.
The stray blank line is gone.

I have local BC3-vs-BC7 comparisons at the same spot, including alpha-tested foliage; I can attach them here if game screenshots are acceptable in the thread. The packing check is not in yet; the header check covers the source/header drift, and I can add the CPU mode-6 decode if you want to hold for it.

Thanks — the new head (`57e53f8b`, rebased on `a5b955fb`) addresses the code points: - **Header drift**: `bc7_encoder` is now in `scripts/host_shaders.py`'s `SHADERS`; the private `scripts/make_bc7_shader.py` is gone; the header was regenerated with glslang 16.6.0 at `vulkan1.1`, and `tests/test_host_shaders.py` passes. The shader and header have the SPDX/provenance opening lines. - **Fallback**: if BC7 is unsupported or its pipeline cannot be made, `bc7_ok` turns the encoder and the textures to BC3 from then on — not RGBA8. - **Power iteration**: if the first step from the diagonal collapses, it restarts once from the covariance row with the largest norm (the R/G anticorrelated, B/A constant case), then the usual iteration. - The stray blank line is gone. I have local BC3-vs-BC7 comparisons at the same spot, including alpha-tested foliage; I can attach them here if game screenshots are acceptable in the thread. The packing check is not in yet; the header check covers the source/header drift, and I can add the CPU mode-6 decode if you want to hold for it.

DFN-master commented

2026-11-11 23:43:33 +00:00

(Migrated from github.com)

Test report — BC7 for big ASTC textures

Machine

Windows 11, 64-bit.
Intel Core i7-11700, 8C/16T, 2.5 GHz base; 32 GB RAM.
NVIDIA GeForce RTX 3080 10 GB; 1920x1080 @ 60 Hz.
Game: v2.0.11-1024-g9bc83387 (source 5d0560d3), 4a53c897 + BC7 + zstd; 4 GB texture budget, RX_VK_COMPACT_PCT=90, RX_VK_READBACK_NO_CATCHUP=1, RX_VK_SYNC_LIVE=1, AF 16x, HDR off, 1080p.

Checks

tests/test_host_shaders.py with glslang 16.6.0 (vulkan1.1): 4 match, including bc7_encoder; the header is regenerated from the .comp, so it cannot drift.
Windows RX_VK_ONLY DLL build: pass, with the BC7 shader and the fallback.
bc7_ok: a GPU without BC7, or a failed BC7 pipeline, falls back to BC3 (not RGBA8); the power iteration restarts from the largest covariance row when the diagonal collapses; the shader/header carry the SPDX/provenance lines.

BC3 vs BC7, same save and camera

Same save, Kokiri Forest/Kakariko entrance, 1920x1080, 300-frame screenshots. The only change is RX_VK_ASTC_BC7=0 vs =1; the log says "big ASTC textures are kept as BC3/BC7" in each run.

The ground, grass/ivy, tree leaves and alpha-tested foliage are visually indistinguishable; I saw no alpha holes, halos or hard edge changes with BC7. The frames are at the same camera; the small lighting difference between the two runs is the game's time/weather, not the encoder.
The BC3 run: 3.9–4.0 GB of textures, 8.1/8.9 GB VRAM; the BC7 run: 3.6 GB, 7.8/8.9 GB. The runs' fps are not a fair encoder comparison (different run timing); the texture/VRAM counts are the relevant part.

The two PNGs are local, and I have not uploaded them. I can provide sanitized copies if game screenshots are acceptable in the thread.

Limits

NVIDIA only; I did not run AMD/Intel, so the encoder's cost there (the register-heavy px[16][4]/two Fits, and the 90-layer arrays) remains unmeasured here.
The packing check is not in yet; the header check covers the source/header drift, and I can add the CPU mode-6 decode if you want to hold for it.
CI: Ubuntu and the BC7/own checks pass; the Windows Python checks fail on main's test_psychov35.py (scanf) and test_pcache_header_read.py (fopen/fseeko), the same on the other PRs.

## Test report — BC7 for big ASTC textures ### Machine - Windows 11, 64-bit. - Intel Core i7-11700, 8C/16T, 2.5 GHz base; 32 GB RAM. - NVIDIA GeForce RTX 3080 10 GB; 1920x1080 @ 60 Hz. - Game: `v2.0.11-1024-g9bc83387` (source 5d0560d3), `4a53c897` + BC7 + zstd; 4 GB texture budget, `RX_VK_COMPACT_PCT=90`, `RX_VK_READBACK_NO_CATCHUP=1`, `RX_VK_SYNC_LIVE=1`, AF 16x, HDR off, 1080p. ### Checks - `tests/test_host_shaders.py` with glslang 16.6.0 (`vulkan1.1`): **4 match**, including `bc7_encoder`; the header is regenerated from the `.comp`, so it cannot drift. - Windows `RX_VK_ONLY` DLL build: pass, with the BC7 shader and the fallback. - `bc7_ok`: a GPU without BC7, or a failed BC7 pipeline, falls back to BC3 (not RGBA8); the power iteration restarts from the largest covariance row when the diagonal collapses; the shader/header carry the SPDX/provenance lines. ### BC3 vs BC7, same save and camera Same save, Kokiri Forest/Kakariko entrance, 1920x1080, 300-frame screenshots. The only change is `RX_VK_ASTC_BC7=0` vs `=1`; the log says "big ASTC textures are kept as BC3/BC7" in each run. - The ground, grass/ivy, tree leaves and alpha-tested foliage are **visually indistinguishable**; I saw no alpha holes, halos or hard edge changes with BC7. The frames are at the same camera; the small lighting difference between the two runs is the game's time/weather, not the encoder. - The BC3 run: 3.9–4.0 GB of textures, 8.1/8.9 GB VRAM; the BC7 run: 3.6 GB, 7.8/8.9 GB. The runs' fps are not a fair encoder comparison (different run timing); the texture/VRAM counts are the relevant part. The two PNGs are local, and I have not uploaded them. I can provide sanitized copies if game screenshots are acceptable in the thread. ### Limits - NVIDIA only; I did not run AMD/Intel, so the encoder's cost there (the register-heavy `px[16][4]`/two `Fit`s, and the 90-layer arrays) remains unmeasured here. - The packing check is not in yet; the header check covers the source/header drift, and I can add the CPU mode-6 decode if you want to hold for it. - CI: Ubuntu and the BC7/own checks pass; the Windows Python checks fail on main's `test_psychov35.py` (scanf) and `test_pcache_header_read.py` (fopen/fseeko), the same on the other PRs.

~~github-sync referenced this pull request 2026-11-12 06:22:54 +00:00~~

chore(main): release 2.2.0-rc.1 #2071

github-sync referenced this pull request

2026-11-12 11:35:59 +00:00

perf(vk): opaque ASTC textures kept as BC1, half the video memory of BC3 #2554

Commenting is not possible because the repository is archived.

No reviewers