MagicBeanDev/Hookshot

Archived

Watch

Fork 0

Run OoT’s TAAU network on AMD RDNA3/RDNA4 matrix hardware (16x16x16 cooperative matrices) #1284

Closed

opened 2026-11-09 20:43:53 +00:00 by Mathias-D06 · 6 comments

Mathias-D06 commented

2026-11-09 20:43:53 +00:00

(Migrated from github.com)

The idea

On an RX 9070 XT (RDNA4, Windows, AMD proprietary driver), Hookshot logs vk: matrix multiplies in software (the cooperative matrices lack 16x8x16 f16). The TAAU network then either runs in software (RX_VK_TAAU_KEYS=-: sharp, but about 43–53 fps instead of a locked 60, with artifacts at object edges) or is replaced by the stand-in upscale (60 fps, but noticeably softer).

AMD’s driver exposes VK_KHR_cooperative_matrix with a 16x16x16 f16 shape but not 16x8x16. vk_coopmat.cpp only accepts MSize 16, NSize 8, KSize 16, so every AMD card falls back to software. Running HMMA.16816 on 16x16x16 matrices (B and the accumulator 16 columns wide) would let RDNA3 and RDNA4 run the network in hardware.

Anything else

Lichii already does this:

zortos293/Lichii PR #583 (“shader: run HMMA on VK_KHR_cooperative_matrix 16x16x16 (RDNA4: Ocarina of Time 41 → 60 fps)”, importing #521 by estevanbs).
PR #657 limits it to AMD GPUs that report integerDotProduct4x8BitPackedMixedSignednessAccelerated (RDNA3 and newer). AMD’s Windows driver also advertises the extension on RDNA2 and older, which have no matrix cores, and building the OoT pipelines crashed there (Lichii #633).

On the same PC (RX 9070 XT, Ryzen 5 7500F), Lichii build 125 logs Guest tensor operations run on 16x16x16 cooperative matrices and the network looks sharp with no edge artifacts. Both projects are GPL-3.0.

Happy to test builds on an RX 9070 XT.

### The idea On an RX 9070 XT (RDNA4, Windows, AMD proprietary driver), Hookshot logs vk: matrix multiplies in software (the cooperative matrices lack 16x8x16 f16). The TAAU network then either runs in software (RX_VK_TAAU_KEYS=-: sharp, but about 43–53 fps instead of a locked 60, with artifacts at object edges) or is replaced by the stand-in upscale (60 fps, but noticeably softer). AMD’s driver exposes VK_KHR_cooperative_matrix with a 16x16x16 f16 shape but not 16x8x16. vk_coopmat.cpp only accepts MSize 16, NSize 8, KSize 16, so every AMD card falls back to software. Running HMMA.16816 on 16x16x16 matrices (B and the accumulator 16 columns wide) would let RDNA3 and RDNA4 run the network in hardware. ### Anything else Lichii already does this: zortos293/Lichii PR #583 (“shader: run HMMA on VK_KHR_cooperative_matrix 16x16x16 (RDNA4: Ocarina of Time 41 → 60 fps)”, importing #521 by estevanbs). PR #657 limits it to AMD GPUs that report integerDotProduct4x8BitPackedMixedSignednessAccelerated (RDNA3 and newer). AMD’s Windows driver also advertises the extension on RDNA2 and older, which have no matrix cores, and building the OoT pipelines crashed there (Lichii #633). On the same PC (RX 9070 XT, Ryzen 5 7500F), Lichii build 125 logs Guest tensor operations run on 16x16x16 cooperative matrices and the network looks sharp with no edge artifacts. Both projects are GPL-3.0. Happy to test builds on an RX 9070 XT.

NotRexed commented

2026-11-09 20:47:15 +00:00

(Migrated from github.com)

Thanks for the detailed pointer, including the RDNA3+ gate. Linking this to #1006, where AMD's matrix-in-software slowdown (the cooperative matrices lack 16x8x16 f16) is tracked.

Thanks for the detailed pointer, including the RDNA3+ gate. Linking this to #1006, where AMD's matrix-in-software slowdown (`the cooperative matrices lack 16x8x16 f16`) is tracked.

kafasam commented

2026-11-10 00:24:48 +00:00

(Migrated from github.com)

RX 6700 XT (RDNA2) data point for this: Adrenalin 26.10.1 (32.0.21045.16010) on Windows also exposes VK_KHR_cooperative_matrix with a 16x16x16 f16 shape and no 16x8x16 — the same mismatch as the RDNA4 cards above. The current probe at vk_coopmat.cpp:375 rejects it and falls back to software, producing the "10-11 networks left out" pattern we've been tracking in #1006 and #1017.

We've verified through a custom build that the game's 11 compute programs (TAAU variants, movie upscaler, gameplay networks) all carry HMHA.16816 cooperative-matrix operations. On our card the software fallback path runs but at significant cost (the TAAU network alone accounts for the "network left out" rejections). If the 16x16x16 shape were accepted alongside 16x8x16, these programs could use the hardware path.

Happy to test any instrumented build or patch that extends the cooperative-matrix probe to accept the 16x16x16 shape on AMD — we have the full build pipeline ready and can turn around test results in under an hour.

RX 6700 XT (RDNA2) data point for this: Adrenalin 26.10.1 (32.0.21045.16010) on Windows also exposes `VK_KHR_cooperative_matrix` with a **16x16x16 f16** shape and no 16x8x16 — the same mismatch as the RDNA4 cards above. The current probe at `vk_coopmat.cpp:375` rejects it and falls back to software, producing the "10-11 networks left out" pattern we've been tracking in #1006 and #1017. We've verified through a custom build that the game's 11 compute programs (TAAU variants, movie upscaler, gameplay networks) all carry HMHA.16816 cooperative-matrix operations. On our card the software fallback path runs but at significant cost (the TAAU network alone accounts for the "network left out" rejections). If the 16x16x16 shape were accepted alongside 16x8x16, these programs could use the hardware path. Happy to test any instrumented build or patch that extends the cooperative-matrix probe to accept the 16x16x16 shape on AMD — we have the full build pipeline ready and can turn around test results in under an hour.

lucasmazzoni45-del commented

2026-11-10 09:47:39 +00:00

(Migrated from github.com)

Data point from RDNA3: RX 7800 XT (Windows 11, Ryzen 7 5800X3D, Vulkan 1.4, driver 0x8001a0) logs the same line on v2.0.12 (release branch build): vk: matrix multiplies in software (the cooperative matrices lack 16x8x16 f16).

What I checked on this PC:

vulkaninfo lists VK_KHR_cooperative_matrix (revision 2) and cooperativeMatrix = true, cooperativeMatrixRobustBufferAccess = true. The summary does not print the supported shapes, so I cannot say from it whether 16x16x16 f16 is the one offered here; the cards above report it, and the same gate in vk_coopmat.cpp is what rejects it here.
On v2.0.12 the default for these cards is the stand-in upscale (taau_key in vk_draw.cpp with mma_software). With RX_VK_TAAU_KEYS=- (the real network, in software) I measured earlier on this card, on an older build and a headless scripted walk, about +1.3 ms of GPU time on 10.6 ms. That is not the 43-53 fps reported for the 9070 XT above, so the cost may differ a lot by scene and card; I have not measured it on v2.0.12.

I can test a build that accepts 16x16x16 on this card (RDNA3, so it would also be a check of the RDNA3 gate in Lichii PR #657) and post the log lines and frame times here.

Data point from RDNA3: **RX 7800 XT** (Windows 11, Ryzen 7 5800X3D, Vulkan 1.4, driver 0x8001a0) logs the same line on v2.0.12 (release branch build): `vk: matrix multiplies in software (the cooperative matrices lack 16x8x16 f16)`. What I checked on this PC: - `vulkaninfo` lists `VK_KHR_cooperative_matrix` (revision 2) and `cooperativeMatrix = true`, `cooperativeMatrixRobustBufferAccess = true`. The summary does not print the supported shapes, so I cannot say from it whether 16x16x16 f16 is the one offered here; the cards above report it, and the same gate in `vk_coopmat.cpp` is what rejects it here. - On v2.0.12 the default for these cards is the stand-in upscale (`taau_key` in `vk_draw.cpp` with `mma_software`). With `RX_VK_TAAU_KEYS=-` (the real network, in software) I measured earlier on this card, on an older build and a headless scripted walk, about +1.3 ms of GPU time on 10.6 ms. That is not the 43-53 fps reported for the 9070 XT above, so the cost may differ a lot by scene and card; I have not measured it on v2.0.12. I can test a build that accepts 16x16x16 on this card (RDNA3, so it would also be a check of the RDNA3 gate in Lichii PR #657) and post the log lines and frame times here.

lucasmazzoni45-del commented

2026-11-10 10:39:33 +00:00

(Migrated from github.com)

Follow-up from the RX 7800 XT (RDNA3, driver 0x8001a0): I tried the 16x16x16 route in a local build of release/v2.0.12, following the approach of Lichii #583 / #657, behind a switch that is off by default. It is not a PR and not working end to end, but the card-side results may save someone time.

What passes on this card: AMD vendor, a 16x16x16 f16 cooperative-matrix shape, integerDotProduct4x8BitPackedMixedSignednessAccelerated (the #657 gate), f16 shaders, required subgroup size 32 on compute. The shader builds and runs on the driver, and a self-test I added (one warp, A*B+C on a 16x16 tile, the game's m16n8k16 fragment layout mapped into a 16x16x16 matrix with columns 8-15 zeroed) can be compared against a CPU reference.

What the self-test found (exact small-integer inputs): 234 of 256 elements are exact. The other 22 are exactly the elements whose true result is 0, and they are off by a few f16 subnormal steps (bits 0x0002..0x0004, so 1e-7..2e-7). With the accumulator in f32 they are different (up to 1.1e-6) but still only on true-zero results. Barrier scope, NonPrivatePointer on the shared-memory accesses, zeroing every tile word and a barrier between multiplies changed nothing: the same elements and values came back each time. So it does not look like a layout or memory bug.

My reading (unproven): the matrix unit's internal accumulation is not an exact IEEE sum (addends aligned or truncated), so a sum like 4 + (-4) can leave a residue of about half an f32 ulp of the operands; for any non-zero result the f16 rounding hides it. If so, a test that demands an exact zero is wrong for this hardware, and the right check is an error bound relative to the operands: |err| <= 2^-18 * (|c| + sum |a*b|) + 2^-24, about 250 times tighter than f16's own epsilon. I have written that bound and a second input set of non-integer values, but have not yet run them on the GPU.

Not known yet: whether the OoT pipelines build and run right through this path on RDNA3, whether the output matches the 16x8 path in the real network, and the speed. I will post the pass/fail line and the frame times here when the test has run. If a maintainer or someone with the Lichii code wants the diff (it touches vk_coopmat.cpp, the HMMA emit code and Sirit), say so and I will share it in whatever form suits.

Follow-up from the RX 7800 XT (RDNA3, driver 0x8001a0): I tried the 16x16x16 route in a local build of `release/v2.0.12`, following the approach of Lichii #583 / #657, behind a switch that is off by default. It is **not a PR and not working end to end**, but the card-side results may save someone time. **What passes on this card:** AMD vendor, a 16x16x16 f16 cooperative-matrix shape, `integerDotProduct4x8BitPackedMixedSignednessAccelerated` (the #657 gate), f16 shaders, required subgroup size 32 on compute. The shader builds and runs on the driver, and a self-test I added (one warp, A*B+C on a 16x16 tile, the game's m16n8k16 fragment layout mapped into a 16x16x16 matrix with columns 8-15 zeroed) can be compared against a CPU reference. **What the self-test found (exact small-integer inputs):** 234 of 256 elements are exact. The other 22 are exactly the elements whose true result is 0, and they are off by a few f16 subnormal steps (bits 0x0002..0x0004, so 1e-7..2e-7). With the accumulator in f32 they are different (up to 1.1e-6) but still only on true-zero results. Barrier scope, NonPrivatePointer on the shared-memory accesses, zeroing every tile word and a barrier between multiplies changed nothing: the same elements and values came back each time. So it does not look like a layout or memory bug. **My reading (unproven):** the matrix unit's internal accumulation is not an exact IEEE sum (addends aligned or truncated), so a sum like 4 + (-4) can leave a residue of about half an f32 ulp of the operands; for any non-zero result the f16 rounding hides it. If so, a test that demands an exact zero is wrong for this hardware, and the right check is an error bound relative to the operands: |err| <= 2^-18 * (|c| + sum |a*b|) + 2^-24, about 250 times tighter than f16's own epsilon. I have written that bound and a second input set of non-integer values, but **have not yet run them on the GPU**. **Not known yet:** whether the OoT pipelines build and run right through this path on RDNA3, whether the output matches the 16x8 path in the real network, and the speed. I will post the pass/fail line and the frame times here when the test has run. If a maintainer or someone with the Lichii code wants the diff (it touches `vk_coopmat.cpp`, the HMMA emit code and Sirit), say so and I will share it in whatever form suits.

lucasmazzoni45-del commented

2026-11-10 11:41:50 +00:00

(Migrated from github.com)

Update from the RX 7800 XT (RDNA3, driver 0x8001a0) with the 16x16x16 route in a local release/v2.0.12 build, behind switches that are off by default. Still not a PR.

With an f32 accumulator the self-test passes (A and B stay f16, the accumulator is f32, converted to f16 at the end of each multiply): integer inputs and a second set of non-integer f16 values (fixed seed, magnitudes 0.01-4), each multiplied twice in a row, all 256 elements within 2^-18 * (|c| + sum |a*b|) + 2^-24 of an exact double-precision CPU result; the matrix unit's own error was at most 6.4e-08 of that bound (1.7% of it). Log line: vk: cooperative matrix 16x16x16 test passed: ... then vk: native matrix multiplies on 16x16x16 f16 cooperative matrices, f32 accumulator. The test is also run against nine deliberately wrong layouts offline (transposed A, shifted rows or k, swapped halves, and so on): each lands at least about 65,000 bounds off, so a wrong mapping cannot pass.

With an f16 accumulator the integer set passes but the random set does not at that bound (118 of 256 beyond it, errors like -7.3008 against -7.3053, i.e. half an f16 step), which is what f16 rounding at each accumulation step gives; the bound for that mode would have to scale with the number of steps, I have not derived it yet. The earlier failures I reported (exact zeros coming back as 1e-7) were this kind of rounding in the exact-zero check, not a layout bug.

Also: the build with the switches on reaches a steady 60 fps at the title (headless), the prepare step lists the same 111 failed programs as without it, and there is no device lost. The movie networks are still left out on AMD by vk_shader.cpp (~1531: not NVIDIA and not kept), regardless of the matrix path.

Not tested: gameplay, the speed of the real TAAU network on the matrix hardware, and whether the picture matches the software path. I will post frame times here after a play session.

Update from the RX 7800 XT (RDNA3, driver 0x8001a0) with the 16x16x16 route in a local `release/v2.0.12` build, behind switches that are off by default. Still not a PR. **With an f32 accumulator the self-test passes** (A and B stay f16, the accumulator is f32, converted to f16 at the end of each multiply): integer inputs and a second set of non-integer f16 values (fixed seed, magnitudes 0.01-4), each multiplied twice in a row, all 256 elements within `2^-18 * (|c| + sum |a*b|) + 2^-24` of an exact double-precision CPU result; the matrix unit's own error was at most 6.4e-08 of that bound (1.7% of it). Log line: `vk: cooperative matrix 16x16x16 test passed: ...` then `vk: native matrix multiplies on 16x16x16 f16 cooperative matrices, f32 accumulator`. The test is also run against nine deliberately wrong layouts offline (transposed A, shifted rows or k, swapped halves, and so on): each lands at least about 65,000 bounds off, so a wrong mapping cannot pass. **With an f16 accumulator** the integer set passes but the random set does not at that bound (118 of 256 beyond it, errors like -7.3008 against -7.3053, i.e. half an f16 step), which is what f16 rounding at each accumulation step gives; the bound for that mode would have to scale with the number of steps, I have not derived it yet. The earlier failures I reported (exact zeros coming back as 1e-7) were this kind of rounding in the exact-zero check, not a layout bug. **Also:** the build with the switches on reaches a steady 60 fps at the title (headless), the prepare step lists the same 111 failed programs as without it, and there is no device lost. The movie networks are still left out on AMD by `vk_shader.cpp` (~1531: not NVIDIA and not kept), regardless of the matrix path. **Not tested:** gameplay, the speed of the real TAAU network on the matrix hardware, and whether the picture matches the software path. I will post frame times here after a play session.

NotRexed commented

2026-11-11 21:31:25 +00:00

(Migrated from github.com)