WIP: feat(vk): built-in shaders from a folder with RX_VK_SHADER_DIR

dyst commented

2026-11-12 23:38:52 +00:00

First-time contributor

Why

Trying an edit to one of the renderer's own shaders (the texture decoders, the upscalers, frame generation, the write-back diff) takes a renderer build today, because their SPIR-V is compiled in. With RX_VK_SHADER_DIR set, the renderer takes each of them from a folder when the file is there, so an edit is a glslang run and a restart.

What changed

renderer: gpu/vk/vk_builtin_spv.{h,cpp} (new): one lookup, builtin_spv(name, words, bytes), behind the BUILTIN_SPV(ARRAY) and BUILTIN_SPV_ARGS(ARRAY) macros. With RX_VK_SHADER_DIR unset it returns the compiled-in words before looking at anything else. Set, it takes each built-in shader from <folder>/<name>.spv when that file is SPIR-V (read_spv: a whole number of words, at least SPIR-V's header, and the magic number first), and logs it once:
```
vk: built-in shader wb_diff_comp from <folder>/wb_diff_comp.spv (6248 bytes; compiled in: 6248) (RX_VK_SHADER_DIR)
```
<name> is the header's (stem_of: clear_vert.spv for clear_vert_spv.h). A missing file means the compiled-in words, without a log line; a file that isn't SPIR-V is named in the log with the reason, and the compiled-in words are used. A file is read once and kept for the process, as the compiled-in words are.
renderer: all 52 compiled-in arrays the renderer uses go through it, in vk_draw.cpp, vk_framegen.cpp, vk_fsr3.cpp, vk_fsr3fi.cpp, vk_fsr3_inputs.cpp, vk_memory.cpp, vk_reactive.cpp, vk_temporal.cpp and vk_texture.cpp. The two FSR 3 pass tables (PassInfo, the PASS macro) carry each array's name for it in spv_name.
renderer: not from a shared longshot_env.txt: a folder of shader code is a _DIR switch, so env_line_allowed already refuses it there. Only the environment the game is started with sets it. The *_spv.h headers stay generated from the sources, as before; the folder is for iteration only.
tests: tests/test_builtin_spv.py (new, game-free): builds the lookup as C++ with a stand-in log and runs it with the folder unset, the file missing, a SPIR-V file, one cut short and one with the wrong first word, in a folder with a space in its name; checks that every compiled-in array of gpu/vk/shaders and host_shaders that gpu/vk uses goes through the lookup (52 here; on main it stops at vk_draw.cpp's seven); and that RX_VK_SHADER_DIR isn't taken from longshot_env.txt.
tests: the three fixtures that compile pieces of vk_framegen.cpp (upscale_independent_test.cpp, framegen_resources_test.cpp, present_source_crop_test.cpp) define BUILTIN_SPV_ARGS beside their stand-in arrays, as the lookup is with the folder unset, so their pointer checks still hold.

Flags and settings

Flag	New or changed	What it does	Default now	Goal: on by default?	Helps on	Doesn't help on	Untested on	Untested, but should help on
`RX_VK_SHADER_DIR`	New	Takes each built-in shader from `<folder>/<name>.spv` when that file is SPIR-V (logged once); a missing or bad file falls back to the compiled-in words. Not taken from `longshot_env.txt`	unset	no (a developer switch, for shader iteration)	-	-	Linux	-

Measurements

Measured on the GitHub version (NotRexed/Hookshot#2878), not yet on Longshot: this PR stays WIP until it is.

Hardware: RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X, 64 GB, Windows 11.
Scene: Kokiri Forest, from a save, headless through the control bridge; that branch built with the Builder's steps.

	this PR (GitHub), `RX_VK_SHADER_DIR` = a folder of the built-in shaders' own words	this PR (GitHub), the same with `wb_diff_comp.spv` two bytes too long
In play, fps	24–28	22–29
Shaders taken from the folder in that scene	`astc_decoder_comp`, `bc_encoder_comp`, `block_linear_unswizzle_2d_comp`, `wb_diff_comp`, each logged once with the same size as the compiled-in one	`wb_diff_comp` refused, the compiled-in one used
Log about a lost device or a crash	nothing	nothing

There is no column for GitHub main or for the switch unset: no such run is recorded. The refusal's log line:

vk: built-in shader wb_diff_comp: <folder>/wb_diff_comp.spv is not a whole number of words, or shorter than SPIR-V's header; the compiled-in one is used

Testing

Tested: the runs under Measurements (on the GitHub version) and the game-free checks this description names. Checked but not measured: the two game runs above (fps ranges, no baseline).

Not tested: Linux (the lookup is standard C++, and paths are fopen's, which is UTF-8 on Windows through the manifest). An edited shader: there is no glslang on the author's PC, so the files were the shipped words, and the run shows the path works, not a visible change.

Needs other testers:

Linux: RX_VK_SHADER_DIR naming a folder of the built-in shaders' own words, expecting each logged as taken from the folder, and a broken file falling back to the compiled-in one (request to open)

Screenshots

Nothing on screen.

Risk and rollback

Unset, builtin_spv returns the compiled-in words before anything else, so a player without the switch runs the same shaders as before; the risk is in the 52 call sites now going through the lookup, where a wrong name or size would hand Vulkan the wrong words. Set, a valid but wrong SPIR-V file in the folder can break rendering: unsetting RX_VK_SHADER_DIR turns it off. Reverting the commit is enough.

Reviewer notes

Carried over from NotRexed/Hookshot#2878 with the move to Longshot: the same change on Longshot's main, with its rename applied. Its review and discussion are there.

Checks run on the GitHub version: every check that reads the changed files (159), through run_all.py on Windows, rebased on GitHub's main of that day: 157 pass; nx2/test_bc_storage_views.py and tests/test_linux_driver_fault_guard.py fail the same way on main there. Only those checks were run, not the whole suite. The checks and the game runs were done on the GitHub version; this branch is the same change with the rename applied (the same files and line counts). Forgejo runs no checks; they were not run again here. The dev-loop proposal's B2 (NotRexed/Hookshot#2636), the Vulkan side of what RX_WGSL_OVERRIDE does for WebGPU.

Issues

None on Longshot. See NotRexed/Hookshot#2636 (B2).

Checklist

For everyone

One topic.
The title is a Conventional Commit (type(scope): description).
Rebased on current main.
python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
Tested in the game, or what wasn't is named under Testing.
Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
Flags table filled in; README / docs/development.md switch reference updated.
Screenshots for a change that shows on screen.
A row in docs/tested-configurations.md for each measured run.
MKWVK_API_VERSION bumped (gpu/vk/vk_api.h's MkwVkHost or MkwVkRenderer changed).
The SPIR-V headers regenerated with glslang (a shader changed).
A game-free test added for the fix, failing on main and passing here.
A check under tests/ that reads changed source updated in this PR, not weakened.
Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
Docs updated.

🤖 Generated with Claude Code

## Why Trying an edit to one of the renderer's own shaders (the texture decoders, the upscalers, frame generation, the write-back diff) takes a renderer build today, because their SPIR-V is compiled in. With `RX_VK_SHADER_DIR` set, the renderer takes each of them from a folder when the file is there, so an edit is a glslang run and a restart. ## What changed - renderer: `gpu/vk/vk_builtin_spv.{h,cpp}` (new): one lookup, `builtin_spv(name, words, bytes)`, behind the `BUILTIN_SPV(ARRAY)` and `BUILTIN_SPV_ARGS(ARRAY)` macros. With `RX_VK_SHADER_DIR` unset it returns the compiled-in words before looking at anything else. Set, it takes each built-in shader from `<folder>/<name>.spv` when that file is SPIR-V (`read_spv`: a whole number of words, at least SPIR-V's header, and the magic number first), and logs it once: ``` vk: built-in shader wb_diff_comp from <folder>/wb_diff_comp.spv (6248 bytes; compiled in: 6248) (RX_VK_SHADER_DIR) ``` `<name>` is the header's (`stem_of`: `clear_vert.spv` for `clear_vert_spv.h`). A missing file means the compiled-in words, without a log line; a file that isn't SPIR-V is named in the log with the reason, and the compiled-in words are used. A file is read once and kept for the process, as the compiled-in words are. - renderer: all 52 compiled-in arrays the renderer uses go through it, in `vk_draw.cpp`, `vk_framegen.cpp`, `vk_fsr3.cpp`, `vk_fsr3fi.cpp`, `vk_fsr3_inputs.cpp`, `vk_memory.cpp`, `vk_reactive.cpp`, `vk_temporal.cpp` and `vk_texture.cpp`. The two FSR 3 pass tables (`PassInfo`, the `PASS` macro) carry each array's name for it in `spv_name`. - renderer: not from a shared `longshot_env.txt`: a folder of shader code is a `_DIR` switch, so `env_line_allowed` already refuses it there. Only the environment the game is started with sets it. The `*_spv.h` headers stay generated from the sources, as before; the folder is for iteration only. - tests: `tests/test_builtin_spv.py` (new, game-free): builds the lookup as C++ with a stand-in log and runs it with the folder unset, the file missing, a SPIR-V file, one cut short and one with the wrong first word, in a folder with a space in its name; checks that every compiled-in array of `gpu/vk/shaders` and `host_shaders` that `gpu/vk` uses goes through the lookup (52 here; on `main` it stops at `vk_draw.cpp`'s seven); and that `RX_VK_SHADER_DIR` isn't taken from `longshot_env.txt`. - tests: the three fixtures that compile pieces of `vk_framegen.cpp` (`upscale_independent_test.cpp`, `framegen_resources_test.cpp`, `present_source_crop_test.cpp`) define `BUILTIN_SPV_ARGS` beside their stand-in arrays, as the lookup is with the folder unset, so their pointer checks still hold. ## Flags and settings | Flag | New or changed | What it does | Default now | Goal: on by default? | Helps on | Doesn't help on | Untested on | Untested, but should help on | |---|---|---|---|---|---|---|---|---| | `RX_VK_SHADER_DIR` | New | Takes each built-in shader from `<folder>/<name>.spv` when that file is SPIR-V (logged once); a missing or bad file falls back to the compiled-in words. Not taken from `longshot_env.txt` | unset | no (a developer switch, for shader iteration) | - | - | Linux | - | ## Measurements Measured on the GitHub version (NotRexed/Hookshot#2878), not yet on Longshot: this PR stays WIP until it is. Hardware: RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X, 64 GB, Windows 11. Scene: Kokiri Forest, from a save, headless through the control bridge; that branch built with the Builder's steps. | | this PR (GitHub), `RX_VK_SHADER_DIR` = a folder of the built-in shaders' own words | this PR (GitHub), the same with `wb_diff_comp.spv` two bytes too long | |---|---|---| | In play, fps | 24–28 | 22–29 | | Shaders taken from the folder in that scene | `astc_decoder_comp`, `bc_encoder_comp`, `block_linear_unswizzle_2d_comp`, `wb_diff_comp`, each logged once with the same size as the compiled-in one | `wb_diff_comp` refused, the compiled-in one used | | Log about a lost device or a crash | nothing | nothing | There is no column for GitHub `main` or for the switch unset: no such run is recorded. The refusal's log line: ``` vk: built-in shader wb_diff_comp: <folder>/wb_diff_comp.spv is not a whole number of words, or shorter than SPIR-V's header; the compiled-in one is used ``` ## Testing **Tested:** the runs under Measurements (on the GitHub version) and the game-free checks this description names. Checked but not measured: the two game runs above (fps ranges, no baseline). **Not tested:** Linux (the lookup is standard C++, and paths are `fopen`'s, which is UTF-8 on Windows through the manifest). An edited shader: there is no glslang on the author's PC, so the files were the shipped words, and the run shows the path works, not a visible change. **Needs other testers:** - Linux: `RX_VK_SHADER_DIR` naming a folder of the built-in shaders' own words, expecting each logged as taken from the folder, and a broken file falling back to the compiled-in one (request to open) ## Screenshots Nothing on screen. ## Risk and rollback Unset, `builtin_spv` returns the compiled-in words before anything else, so a player without the switch runs the same shaders as before; the risk is in the 52 call sites now going through the lookup, where a wrong name or size would hand Vulkan the wrong words. Set, a valid but wrong SPIR-V file in the folder can break rendering: unsetting `RX_VK_SHADER_DIR` turns it off. Reverting the commit is enough. ## Reviewer notes Carried over from [NotRexed/Hookshot#2878](https://github.com/NotRexed/Hookshot/pull/2878) with the move to Longshot: the same change on Longshot's `main`, with its rename applied. Its review and discussion are there. Checks run on the GitHub version: every check that reads the changed files (159), through `run_all.py` on Windows, rebased on GitHub's `main` of that day: 157 pass; `nx2/test_bc_storage_views.py` and `tests/test_linux_driver_fault_guard.py` fail the same way on `main` there. Only those checks were run, not the whole suite. The checks and the game runs were done on the GitHub version; this branch is the same change with the rename applied (the same files and line counts). Forgejo runs no checks; they were not run again here. The dev-loop proposal's B2 ([NotRexed/Hookshot#2636](https://github.com/NotRexed/Hookshot/pull/2636)), the Vulkan side of what `RX_WGSL_OVERRIDE` does for WebGPU. ## Issues None on Longshot. See [NotRexed/Hookshot#2636](https://github.com/NotRexed/Hookshot/pull/2636) (B2). ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [ ] Rebased on current `main`. - [ ] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [ ] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [ ] Flags table filled in; README / docs/development.md switch reference updated. - [ ] Screenshots for a change that shows on screen. - [ ] A row in `docs/tested-configurations.md` for each measured run. - [ ] `MKWVK_API_VERSION` bumped (`gpu/vk/vk_api.h`'s `MkwVkHost` or `MkwVkRenderer` changed). - [ ] The SPIR-V headers regenerated with glslang (a shader changed). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [ ] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. - [ ] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dyst added 1 commit

2026-11-12 23:38:52 +00:00

feat(vk): built-in shaders from a folder with RX_VK_SHADER_DIR 03fff16cf2

With RX_VK_SHADER_DIR=<folder>, each of the renderer's built-in shaders
is taken from <folder>/<name>.spv (clear_vert.spv for clear_vert_spv.h)
when that file is SPIR-V, and the log names it; otherwise, and unset,
the compiled-in words. A shader edit can then be tried by compiling it
into the folder and starting the game again, without a renderer build.
The dev-loop proposal's B2 (#2636).

One lookup (gpu/vk/vk_builtin_spv.h, BUILTIN_SPV) serves the 52
compiled-in arrays the renderer uses; the FSR 3 tables carry each
array's name. The folder is not taken from longshot_env.txt (a _DIR
switch). tests/test_builtin_spv.py builds the lookup and runs it against
made-up files, and checks that no array is used around it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

dyst changed title from ~~feat(vk): built-in shaders from a folder with RX_VK_SHADER_DIR~~ to WIP: feat(vk): built-in shaders from a folder with RX_VK_SHADER_DIR

2026-11-13 01:13:56 +00:00

dyst commented

2026-11-13 01:13:57 +00:00

Author

First-time contributor

Back to WIP under #48's process: this was carried over before the new template. Its description will be brought to the template and measured on Longshot (main against this PR, at named commits) when it's wanted for review; ask here if you'd like it sooner.

dyst commented

2026-11-13 01:58:22 +00:00

Author

First-time contributor

Description brought to Longshot's PR template (#48). Still WIP: its numbers are from the GitHub version, and it isn't rebased on current main; it gets measured on Longshot (main against this PR, at named commits) before it is marked ready.

dyst commented

2026-11-13 02:00:38 +00:00

Author

First-time contributor

Description moved to the template's Testing section and whole checklist (#72): what was tested, what wasn't, and what needs other testers (their test requests to open while this is WIP).

This pull request is marked as a work in progress.

This branch is out-of-date with the base branch

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'feat/builtin-shader-dir:dyst-feat/builtin-shader-dir'

git switch 'dyst-feat/builtin-shader-dir'

Rows
Columns

WIP: feat(vk): built-in shaders from a folder with RX_VK_SHADER_DIR #33

Why

What changed

Flags and settings

Measurements

Testing

Screenshots

Risk and rollback

Reviewer notes

Issues

Checklist

For everyone

If applicable

Checkout