MagicBeanDev/Hookshot

Archived

Watch

Fork 0

perf(vk): hoist block-invariant ASTC decoder work #2829

Open

github-sync wants to merge 3 commits from github/pr/2829 into main

github-sync commented

2026-11-12 15:49:13 +00:00

Owner

@Arkex-ai opened this pull request on GitHub · 2026-11-12 15:36 UTC · original

Summary

The ASTC compute decoder repeats partition and weight-grid calculations for every pixel even though their inputs are constant for the block. Moving that work out of the pixel loop reduced measured ASTC-upload GPU time by a median 3.29% in three matched early-Kokiri pairs on an RTX 5070 Ti. This is an upload-time result, not an FPS claim.

Technical changes

Compute partition hashes/scales and weight-grid scaling once per block; replace the endpoint-encoding scan with an equivalent boundary search. Keep workgroups, bindings, push constants, HDR branches, tiling and upload/BC policy unchanged.
Regenerate the shader header using the official Vulkan 1.1 recipe. Preserve the Eden/yuzu notices and credit Lichii's change, including justr4mzi/SmileyTatsu's work.
Add source-coupled CPU regression and an authored full-shader Vulkan comparison. Normal CI only compiles the Vulkan fixture; GPU execution is explicit. Method, commands and limits.

Validation

Current head: 0dde4fb5331e0a31f298d76ab7f531f1934eb075, rebased onto main 15d4ae8c; fresh merge-tree audit against 3c2d80cf is clean. The worklist conflict retains main's updated Intel assessment and the ASTC measured row, without restoring the superseded Intel row. All seven other proposal-file blobs, including shader/header, CPU regression and Vulkan fixture, are identical to 37c95886. CPU regression and Vulkan fixture compilation pass on Windows and Linux at this head. New hosted CI is pending. This base includes main's shared source-check repairs (#2861/#2883). All three affected fixtures pass locally on Windows (MinGW) and Linux (Clang18); hosted MSVC CRT coverage remains separate. No fresh GPU execution, native/game run or performance measurement is claimed after the rebase; the earlier results below remain pinned to their tested source. Earlier full source suites at 9ddc3231: Linux 472 passed, 1 failed, 24 skipped; Windows 459 passed, 3 failed, 35 skipped. The Linux nightly-concurrency fixture fails intermittently with HTTP404/checksum mismatch on both exact main (4/10) and candidate (5/10); its test/release/workflow files are unchanged. All three Windows failures reproduce on exact main (NUP symlink privilege, bug-report decoding, input-fixture unused global). Original failures are retained; no full-green local-suite claim.

Earlier-head CI (`d2a30307`)	Result / run
Game-free source regressions	success
Windows builder	success
Linux AppImage	success
PR title	success

CPU regression: 3,935,232 partition positions and 8,448 endpoint choices per OS; wrong-tie/boundary controls rejected.
Actual shipped shader: 88 authored cases pass on Windows/NVIDIA and explicit Linux llvmpipe (software only). Wrong-zero and wrong-tie GPU controls fail on pixel differences. Khronos validation was unavailable locally; the independent report below covers lavapipe.
Independent validation report: mooki000 ran the 88 authored cases at 515b66ca on Arch/lavapipe with Khronos validation (GCC16.2 and Clang23.1): 0 errors, 0 warnings. Baseline/baseline and zero-store rejection controls also pass. The shader/header and fixture blobs are unchanged here; this remains an attributed earlier-head report, not a d8 hardware/game/conformance run.
Implementation 3d7c7a22 has the identical shader/header: matching native Windows build and an uninstrumented baseline/candidate Kokiri load both pass. Eight further processes with identical private timestamp instrumentation exit normally and preserve the checkpoint.
Paired matched upload GPU times: 224.279 â†’ 218.817 ms, 224.697 â†’ 217.310 ms, 223.363 â†’ 215.190 ms (AB/BA/AB); A/A 222.197 â†’ 222.769 ms. Matched coverage: baseline 100%, candidate â‰¥99.9976% of upload time. Dynamic GPU clocks; same host, input snapshot/shape/count groups, options and cache seed. The measured span includes barriers, BC conversion and copies. Individual runs.

Limits: one Windows/NVIDIA machine and early Kokiri, no measured FPS or uninstrumented startup improvement. Linux hardware, AMD/Intel, campaign and HDR remain untested. An independent Arm-reference discrepancy exists identically in baseline and candidate; this PR does not claim to fix ASTC conformance. The private profiler and game inputs are excluded.

> [@Arkex-ai](https://github.com/Arkex-ai) opened this pull request on GitHub · 2026-11-12 15:36 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2829) ## Summary The ASTC compute decoder repeats partition and weight-grid calculations for every pixel even though their inputs are constant for the block. Moving that work out of the pixel loop reduced measured ASTC-upload GPU time by a median **3.29%** in three matched early-Kokiri pairs on an RTX 5070 Ti. This is an upload-time result, not an FPS claim. ## Technical changes - Compute partition hashes/scales and weight-grid scaling once per block; replace the endpoint-encoding scan with an equivalent boundary search. Keep workgroups, bindings, push constants, HDR branches, tiling and upload/BC policy unchanged. - Regenerate the shader header using the official Vulkan 1.1 recipe. Preserve the Eden/yuzu notices and credit [Lichii's change](https://github.com/zortos293/Lichii/commit/d590d51820f99447bd7056920e28eaeab510e4f1), including justr4mzi/SmileyTatsu's work. - Add source-coupled CPU regression and an authored full-shader Vulkan comparison. Normal CI only compiles the Vulkan fixture; GPU execution is explicit. [Method, commands and limits](https://github.com/Arkex-ai/Hookshot/blob/9ddc32313d682fab7b02e670c75b8eafe9d791fa/docs/astc-decoder.md). ## Validation Current head: `0dde4fb5331e0a31f298d76ab7f531f1934eb075`, rebased onto main `15d4ae8c`; fresh merge-tree audit against `3c2d80cf` is clean. The worklist conflict retains main's updated Intel assessment and the ASTC measured row, without restoring the superseded Intel row. All seven other proposal-file blobs, including shader/header, CPU regression and Vulkan fixture, are identical to `37c95886`. CPU regression and Vulkan fixture compilation pass on Windows and Linux at this head. New hosted CI is pending. This base includes main's shared source-check repairs (#2861/#2883). All three affected fixtures pass locally on Windows (MinGW) and Linux (Clang18); hosted MSVC CRT coverage remains separate. No fresh GPU execution, native/game run or performance measurement is claimed after the rebase; the earlier results below remain pinned to their tested source. Earlier full source suites at `9ddc3231`: Linux **472 passed, 1 failed, 24 skipped**; Windows **459 passed, 3 failed, 35 skipped**. The Linux nightly-concurrency fixture fails intermittently with HTTP404/checksum mismatch on both exact main (4/10) and candidate (5/10); its test/release/workflow files are unchanged. All three Windows failures reproduce on exact main (NUP symlink privilege, bug-report decoding, input-fixture unused global). Original failures are retained; no full-green local-suite claim. | Earlier-head CI (`d2a30307`) | Result / run | |---|---| | Game-free source regressions | [success](https://github.com/NotRexed/Hookshot/actions/runs/43251708257) | | Windows builder | [success](https://github.com/NotRexed/Hookshot/actions/runs/43251708308) | | Linux AppImage | [success](https://github.com/NotRexed/Hookshot/actions/runs/43251708054) | | PR title | [success](https://github.com/NotRexed/Hookshot/actions/runs/43251811451) | - CPU regression: 3,935,232 partition positions and 8,448 endpoint choices per OS; wrong-tie/boundary controls rejected. - Actual shipped shader: 88 authored cases pass on Windows/NVIDIA and explicit Linux llvmpipe (software only). Wrong-zero and wrong-tie GPU controls fail on pixel differences. Khronos validation was unavailable locally; the independent report below covers lavapipe. - **Independent validation report:** [mooki000](https://git.magicbean.dev/MagicBeanDev/Hookshot/pulls/2829) ran the 88 authored cases at `515b66ca` on Arch/lavapipe with Khronos validation (GCC16.2 and Clang23.1): **0 errors, 0 warnings**. Baseline/baseline and zero-store rejection controls also pass. The shader/header and fixture blobs are unchanged here; this remains an attributed earlier-head report, not a d8 hardware/game/conformance run. - Implementation `3d7c7a22` has the identical shader/header: matching native Windows build and an uninstrumented baseline/candidate Kokiri load both pass. Eight further processes with identical private timestamp instrumentation exit normally and preserve the checkpoint. - Paired matched upload GPU times: **224.279 â†’ 218.817 ms**, **224.697 â†’ 217.310 ms**, **223.363 â†’ 215.190 ms** (AB/BA/AB); A/A **222.197 â†’ 222.769 ms**. Matched coverage: baseline 100%, candidate â‰¥99.9976% of upload time. Dynamic GPU clocks; same host, input snapshot/shape/count groups, options and cache seed. The measured span includes barriers, BC conversion and copies. [Individual runs](https://github.com/Arkex-ai/Hookshot/blob/9ddc32313d682fab7b02e670c75b8eafe9d791fa/docs/tested-configurations.md). Limits: one Windows/NVIDIA machine and early Kokiri, no measured FPS or uninstrumented startup improvement. Linux hardware, AMD/Intel, campaign and HDR remain untested. An independent Arm-reference discrepancy exists identically in baseline and candidate; this PR does not claim to fix ASTC conformance. The private profiler and game inputs are excluded.

github-sync added 3 commits

2026-11-12 15:49:13 +00:00

perf(vk): hoist invariant ASTC decoder work 3d7c7a221d

Compute partition hashes and weight scaling once per block, and use an
equivalent endpoint-encoding boundary search. Preserve Hookshot tiling,
bounds, formats and upload policy. Regenerate the Vulkan 1.1 shader header
and compare the actual partition/search functions against frozen references.

Adapted from Lichii #296 by justr4mzi, based on SmileyTatsu87's changes;
retain the existing Eden/yuzu SPDX notices and source attribution.

Assisted-by: OpenAI Codex

test(vk): add optional authored ASTC GPU comparison 9ddc32313d

Compare the shipped module against the pinned GPL baseline without game inputs. Keep CI compile-only; record the bounded native upload measurements and remaining hardware/conformance limits.

Assisted-by: OpenAI Codex

test(vk): allow standard fopen with Windows MSVC headers d2a3030709

Keep strict warnings while using the same CRT compatibility define as existing native font tests. This fixes the Windows source-shard compilation failure; no shader or runtime change.

Assisted-by: OpenAI Codex

github-sync force-pushed github/pr/2829 from d2a3030709 to 515b66ca45

2026-11-12 16:54:34 +00:00

Compare

github-sync referenced this pull request

2026-11-12 17:57:29 +00:00

ci: execute authored Vulkan fixtures on lavapipe #2866

github-sync force-pushed github/pr/2829 from 515b66ca45 to d8f010c0a4

2026-11-12 17:57:50 +00:00

Compare

github-sync commented

2026-11-12 17:59:42 +00:00

Author

Owner

@mooki000 commented on GitHub · 2026-11-12 17:21 UTC · original

The validation-layer pass you did not have yet, as promised in the team room: the 88-case fixture passes on lavapipe under the Khronos validation layer with 0 errors and 0 warnings, at 515b66ca.

Run (`--run --device llvmpipe`)	Result	Validation errors	Warnings
this PR's shader against the pinned baseline, GCC 16.2 build of the fixture	PASS, 88 cases	0	0
the same, Clang 23.1 build	PASS, 88 cases	0	0
the baseline as both inputs (control for the fixture's own Vulkan use)	PASS, 88 cases	0	0

Setup: Arch Linux, lavapipe from Mesa 26.2.5 (LLVM 23.1.3), validation layer 1.4.363, core checks on and the duplicate-message limit off. The loader was given only lavapipe's manifest, and a device listing showed llvmpipe as the one device, so no hardware could be picked by accident. The layer's own start-up record is in each counted run.
The only records are two informational ones per run: the layer saying it is enabled, and its note that its shader cache file does not exist yet in a fresh cache folder. Both appear in the baseline-as-both run too.
The fixture can fail: your documented zero-store mutation (imageStore(..., vec4(0.0))), compiled with the repository's glslang recipe and passed through --candidate-spirv, is rejected at the first case (17 != 0).
Also pass: tests/test_astc_decoder.py and the compile-only form, with both compilers.

For the shared CI job (the lavapipe fixtures branch): this fits as one more entry in its fixture list. One thing it needs there: the hosted checkout is shallow and does not contain 51da0464, so the job has to fetch that one commit (or take the baseline through --baseline-header). I have that addition as an untested patch and will bring it once both are in.

Not tested: real GPUs, Windows, other lavapipe versions, the game, performance. This shows the two shaders agree on your authored cases and that the fixture's Vulkan use is clean, not ASTC conformance.

> [@mooki000](https://github.com/mooki000) commented on GitHub · 2026-11-12 17:21 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2829#issuecomment-6830774666) The validation-layer pass you did not have yet, as promised in the team room: **the 88-case fixture passes on lavapipe under the Khronos validation layer with 0 errors and 0 warnings**, at `515b66ca`. | Run (`--run --device llvmpipe`) | Result | Validation errors | Warnings | |---|---|---|---| | this PR's shader against the pinned baseline, GCC 16.2 build of the fixture | PASS, 88 cases | 0 | 0 | | the same, Clang 23.1 build | PASS, 88 cases | 0 | 0 | | the baseline as both inputs (control for the fixture's own Vulkan use) | PASS, 88 cases | 0 | 0 | - **Setup:** Arch Linux, lavapipe from Mesa 26.2.5 (LLVM 23.1.3), validation layer 1.4.363, core checks on and the duplicate-message limit off. The loader was given only lavapipe's manifest, and a device listing showed llvmpipe as the one device, so no hardware could be picked by accident. The layer's own start-up record is in each counted run. - **The only records** are two informational ones per run: the layer saying it is enabled, and its note that its shader cache file does not exist yet in a fresh cache folder. Both appear in the baseline-as-both run too. - **The fixture can fail:** your documented zero-store mutation (`imageStore(..., vec4(0.0))`), compiled with the repository's glslang recipe and passed through `--candidate-spirv`, is rejected at the first case (`17 != 0`). - **Also pass:** `tests/test_astc_decoder.py` and the compile-only form, with both compilers. For the shared CI job (the lavapipe fixtures branch): this fits as one more entry in its fixture list. One thing it needs there: the hosted checkout is shallow and does not contain `51da0464`, so the job has to fetch that one commit (or take the baseline through `--baseline-header`). I have that addition as an untested patch and will bring it once both are in. Not tested: real GPUs, Windows, other lavapipe versions, the game, performance. This shows the two shaders agree on your authored cases and that the fixture's Vulkan use is clean, not ASTC conformance.

github-sync commented

2026-11-12 17:59:42 +00:00

Author

Owner

@Arkex-ai commented on GitHub · 2026-11-12 17:49 UTC · original

Thanks, this fills the validation-layer gap. I added your result to the description with the tested 515b66ca attribution; the shader/header and fixture are unchanged at d8f010c0. Keeping the baseline commit fetch in your existing lavapipe lane makes sense for shallow checkouts. The current Windows source failure is in the unchanged write-back-shadow fixture; its exact job is linked separately in Validation.

> [@Arkex-ai](https://github.com/Arkex-ai) commented on GitHub · 2026-11-12 17:49 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2829#issuecomment-6831452124) Thanks, this fills the validation-layer gap. I added your result to the description with the tested `515b66ca` attribution; the shader/header and fixture are unchanged at `d8f010c0`. Keeping the baseline commit fetch in your existing lavapipe lane makes sense for shallow checkouts. The current Windows source failure is in the unchanged write-back-shadow fixture; its exact job is linked separately in Validation.

github-sync force-pushed github/pr/2829 from d8f010c0a4 to 0dde4fb533

2026-11-12 19:01:01 +00:00

Compare

Commenting is not possible because the repository is archived.

No reviewers