feat: Save my pipeline list, the zip and filled-in report for the "Submit your pipelines" issue in one click #2447

Merged

hckrman101 merged 14 commits from feat/share-pipeline-list into main

2026-11-11 22:08:59 +00:00

hckrman101 commented

2026-11-11 19:29:34 +00:00

(Migrated from github.com)

Sending a pipeline list to #2034 today means finding hookshot_pipeline_states.bin next to the executable, zipping it by hand and filling in a seven-line template from settings spread over the menu and the log. Few players get through that, and every list sent removes first-load stutter for everyone, so this makes it one click in the game and in both builders.

What it does

Save my pipeline list (in-game menu, Debug tab, first row under Developer; and the launcher page of both Hookshot Builders):

Checks hookshot_pipeline_states.bin the way nx2/pipeline_states.py --check checks a list: there, a format this build reads (HKSTATE2 or HKSTATE1, right record size), whole records only, at least one pipeline. If it isn't, it says why in plain words and writes nothing.
Writes hookshot-pipelines-<date>.zip to the Downloads folder (Windows: the Downloads known folder; Linux: XDG_DOWNLOAD_DIR from user-dirs.dirs, else ~/Downloads, else home). The zip holds exactly two files: the list as the game wrote it, and hookshot_pipeline_report.txt, which is the issue's template filled in:
- Hookshot version
- OS, GPU and driver (the driver as it names itself: "NVIDIA 580.95.05", "radv Mesa 25.2.4")
- Output resolution (Handheld profile and Ultrawide noted)
- Upscaler + render resolution, with the 3D's actual size ("NVIDIA DLSS, Quality 67% (1707x960)")
- Frame generation / HDR
- Mods or texture packs (the mods folder, by the same rule host/mods.c loads them: folders with romfs/exefs, -/_ ones skipped)
- Where you played: left empty for the player
- Pipelines: the count and list version
  No path, user name, save, cache, options file or log goes in.
The list holds only pipelines the game drew on this PC: predicted pipelines and the shipped list are never written to it.
On a worker thread, so the frame never waits on it, it copies the template to the clipboard (Windows clipboard; Linux wl-copy, else xclip/xsel; the Linux builder uses Tk's), shows the folder (Explorer with the zip selected; xdg-open on Linux) and opens #2034 in the browser.
Says what happened: the file name and pipeline count, whether the report was copied, and "attach the zip to a comment, then paste the report and say where you played". If the browser could not be opened it shows the issue's address instead.

While that runs, another press is ignored. Nothing is ever uploaded. In a run without a window (HOS_HEADLESS) the menu saves the zip but opens nothing and leaves the clipboard alone.

How

gpu/vk/pipeline_share.h: the check, the template, a small stored-zip writer (CRC-32) and the user-dirs.dirs reader, as plain functions with no Vulkan, so a test compiles them.
gpu/vk/vk_menu.cpp: the row (Debug tab, drawn by that tab's own layout), share_pipeline_list() (values from the live settings through the menu's own tx_* functions, so they read exactly as the menu shows them) and the platform pieces (Downloads folder, clipboard, opening the folder and the browser). On Windows, shell32/ole32 (SHGetKnownFolderPath, ShellExecuteW, CoInitializeEx) are loaded from system32 when the row is used, so hookshot_vk.dll's imports stay within the DLSS workflow's allowlist. COM is initialised on the worker for ShellExecuteW. On Linux, desktop programs are started with only their three standard files (close-on-exec clipboard pipe, addclosefrom_np). The zip is written to a fresh .new file (O_EXCL|O_NOFOLLOW / CREATE_NEW) and moved over the old one. gpu/vk/vk_draw.cpp state_list_snapshot() reads the list under the lock the recorder writes it with, so a record being written at that moment is never half in the zip.
gpu/vk/vk_main.cpp logs vk: driver: <name> <info> once at start, so the builders can report the driver too (older logs: decoded from the version number).
nx2/pipeline_share.py: the same zip for the builders (the list read once, so the bytes checked are the bytes zipped), from the game folder (version from the last log or hookshot_version.txt, GPU and driver from the last log, settings from hookshot_options.ini, mods from mods/). The Linux builder imports it; the Windows builder runs it with --json with its own Python, then does the clipboard, Explorer and browser itself. It also works by hand: python -I nx2/pipeline_share.py <game folder>.
Docs: readme (new "Send your pipeline list" section under "Want to help?", the menu row, the stutter troubleshooting row), assets/pipeline_states/README.md, docs/development.md.

Tests

tests/test_pipeline_share.py (game-free, picked up by run_all.py):

the zip from both the C++ header and the Python holds exactly hookshot_pipeline_states.bin (byte-identical) and hookshot_pipeline_report.txt, and nothing else from a game folder that also has saves, caches, the options file and the log; the CRCs check;
the template has every field in the issue's order, the expected values from a synthetic options file, log and mods folder, "where you played" empty, and no path of the player's;
missing, not-a-list, cut-short and empty lists are refused with the reason and no zip is written; C++ and Python agree on every case;
the C++ and Python field lists are the same, and the Python's names for the upscalers, render resolution shares, output sizes, frame generation and HDR values match vk_menu.cpp / upscaler_rows.h;
the list holds only pipelines the game drew: state_record() is reached only from draw() through graphics_pipeline() and pipeline_note(), never from the precompile, predicted pipelines or the shipped list;
the C++ and Python Output resolution lines agree (Handheld profile, the game's own size, Ultrawide);
the menu row saves on Enter (not the Controls page catch-all), checks before zipping, opens nothing under HOS_HEADLESS, opens the browser on a worker with COM initialised, gives desktop programs none of the game's files, and uploads nothing;
the Linux builder's action, with stand-ins for the clipboard and xdg-open: the zip, then copy, folder, issue; a refusal opens nothing.

Full python -I -B tests/run_all.py on Linux, on the final tree (main merged in): 401 passed, 0 failed, 12 skipped (no dotnet, PIL or some headers here).

Checked by hand

Game (Linux, RTX 3090, NVIDIA 620.70.06, headless run, HOS_HEADLESS=1 HOS_NO_SDL=1, title screen; build of this branch with main merged in, v2.1.0-rc.3-120-g5df019f): the menu opened with RX_VK_MENU_SCRIPT, then Debug tab, Save my pipeline list, Enter. The row changed from Save to Saved, and the help box said where the zip went and what to do next. The log said menu: Save my pipeline list: hookshot-pipelines-2026-11-11.zip, 672 pipelines, in Downloads. The zip held exactly hookshot_pipeline_states.bin (byte-identical to the game folder's) and hookshot_pipeline_report.txt:
```
- Hookshot version: v2.1.0-rc.3-120-g5df019f (source 9d480ad7)
- OS, GPU and driver: Linux (CachyOS, kernel 7.3.3-1-cachyos); NVIDIA GeForce RTX 3090, driver NVIDIA 620.70.06
- Output resolution: 2048x1152 (the game's own)
- Upscaler + render resolution: Game's own, Auto (dynamic)
- Frame generation / HDR: Off / Off
- Mods or texture packs: none
- Where you played (areas, dungeons, age, night/rain, ...):
- Pipelines: 672 (version 2)
```
The run was headless, so as designed it opened nothing.
Linux builder: the launcher page in a nested headless sway, the button pressed through Tk: the zip landed in the Downloads folder from user-dirs.dirs, xdg-open was asked for the folder and the issue, the status line says what happened (screenshots).
Windows builder: compiles (dotnet build -c Debug -p:EnableWindowsTargeting=true -warnaserror, .NET 8 SDK on Linux, 0 warnings). Not run on Windows: the clipboard, Explorer and browser calls there are untested.
Windows game: vk_menu.cpp and vk_main.cpp compile with llvm-mingw. vk_menu.o imports only kernel32, user32 and UCRT functions; shell32 and ole32 are loaded at run time. Not run on Windows.

Screenshots

In-game menu, Debug tab, the row selected:
After Enter: the row says Saved, and the help box says where the zip went and what to do:
Linux builder launcher page after pressing the button:

Suggested new body for #2034 (not edited)

When Hookshot meets a pipeline it hasn't built on your PC yet, it builds it on the spot, which causes a stutter, or a short white wash on the first field load. Releases ship a list of known pipelines that the first start builds ahead of time. The list only covers what someone has played, so send us the list your game wrote while you played and we'll merge it into the next release.

Is it safe? Yes. The file holds no game code, game data or personal info: only shader hashes, formats and draw state. See assets/pipeline_states/README.md.

How to submit

Play on a recent build (a build from main), see https://github.com/hckrman101/HookshotDownloads/releases
Save your list:
- In the game: open the menu (Insert or F1), go to the Debug tab and choose Save my pipeline list (under Developer).
- Or in Hookshot Builder: on the game folder's launcher page, press Save my pipeline list.
It saves hookshot-pipelines-<date>.zip in your Downloads folder, copies a filled-in report to your clipboard and opens this issue.
Comment here: drag the zip in, paste the report, and fill in where you played.

Nothing is uploaded for you, and the zip holds only hookshot_pipeline_states.bin and the report (version, OS, GPU and driver, output and render resolution, upscaler, frame generation, HDR, mods). If your list is missing or damaged, it tells you and saves nothing.

Settings matter, so lists from different GPUs, upscalers and resolutions are especially welcome, as are areas few people reach (late dungeons, bosses, night, rain, underwater).

On a build without Save my pipeline list

Zip hookshot_pipeline_states.bin from next to the Hookshot executable (only that file: no hookshot_shaders.bin, saves or anything else from the game folder), drag the zip into a comment here and fill in:

- Hookshot version:
- OS, GPU and driver:
- Output resolution:
- Upscaler + render resolution (game's / DLSS / FSR 3.1 / FSR 4; Quality / Balanced / ...):
- Frame generation / HDR: on / off
- Mods or texture packs:
- Where you played (areas, dungeons, age, night/rain, ...):

Or, from a clone of the repository: python -I nx2/pipeline_share.py <game folder> makes the same zip and report.

What happens next

We merge each list with nx2/pipeline_states.py (each pipeline kept once) and note its source in assets/pipeline_states/. Lists that only repeat known pipelines add nothing, and that's fine.

Thanks!

(Changes from the current body: the one-click steps lead; the old manual steps and template move into a collapsed section; the AI-agent paragraph is dropped because the button does that job; the safety link points at assets/pipeline_states/README.md, because the current assets/pipeline_states.md link is broken.)

Not tested

The non-headless path in the game (the worker): clipboard (wl-copy/xclip/xsel; on Windows, CF_UNICODETEXT), opening the folder and opening the browser. Test runs are headless by rule, so nothing was opened.
Anything on Windows: the game's menu action, the WPF builder's button (it compiles only), Explorer /select and the Downloads known folder.
The Linux AppImage. xdg-open inherits the AppImage's environment there, as the game's existing update prompt already does.
The refusal message in the game (missing, empty or damaged list). It is covered by the game-free test for the same C++ code, not by a run.
The Python/Qt Windows builder (builder/py) has no button yet. It could call nx2/pipeline_share.py the same way.

Sending a pipeline list to #2034 today means finding `hookshot_pipeline_states.bin` next to the executable, zipping it by hand and filling in a seven-line template from settings spread over the menu and the log. Few players get through that, and every list sent removes first-load stutter for everyone, so this makes it one click in the game and in both builders. ## What it does **Save my pipeline list** (in-game menu, Debug tab, first row under Developer; and the launcher page of both Hookshot Builders): 1. Checks `hookshot_pipeline_states.bin` the way `nx2/pipeline_states.py --check` checks a list: there, a format this build reads (HKSTATE2 or HKSTATE1, right record size), whole records only, at least one pipeline. If it isn't, it says why in plain words and writes nothing. 2. Writes `hookshot-pipelines-<date>.zip` to the Downloads folder (Windows: the Downloads known folder; Linux: `XDG_DOWNLOAD_DIR` from `user-dirs.dirs`, else `~/Downloads`, else home). The zip holds exactly two files: the list as the game wrote it, and `hookshot_pipeline_report.txt`, which is the issue's template filled in: - Hookshot version - OS, GPU and driver (the driver as it names itself: "NVIDIA 580.95.05", "radv Mesa 25.2.4") - Output resolution (Handheld profile and Ultrawide noted) - Upscaler + render resolution, with the 3D's actual size ("NVIDIA DLSS, Quality 67% (1707x960)") - Frame generation / HDR - Mods or texture packs (the mods folder, by the same rule `host/mods.c` loads them: folders with `romfs`/`exefs`, `-`/`_` ones skipped) - Where you played: left empty for the player - Pipelines: the count and list version No path, user name, save, cache, options file or log goes in. The list holds only pipelines the game drew on this PC: predicted pipelines and the shipped list are never written to it. 3. On a worker thread, so the frame never waits on it, it copies the template to the clipboard (Windows clipboard; Linux `wl-copy`, else `xclip`/`xsel`; the Linux builder uses Tk's), shows the folder (Explorer with the zip selected; `xdg-open` on Linux) and opens #2034 in the browser. 4. Says what happened: the file name and pipeline count, whether the report was copied, and "attach the zip to a comment, then paste the report and say where you played". If the browser could not be opened it shows the issue's address instead. While that runs, another press is ignored. Nothing is ever uploaded. In a run without a window (`HOS_HEADLESS`) the menu saves the zip but opens nothing and leaves the clipboard alone. ## How - `gpu/vk/pipeline_share.h`: the check, the template, a small stored-zip writer (CRC-32) and the `user-dirs.dirs` reader, as plain functions with no Vulkan, so a test compiles them. - `gpu/vk/vk_menu.cpp`: the row (Debug tab, drawn by that tab's own layout), `share_pipeline_list()` (values from the live settings through the menu's own `tx_*` functions, so they read exactly as the menu shows them) and the platform pieces (Downloads folder, clipboard, opening the folder and the browser). On Windows, `shell32`/`ole32` (`SHGetKnownFolderPath`, `ShellExecuteW`, `CoInitializeEx`) are loaded from system32 when the row is used, so `hookshot_vk.dll`'s imports stay within the DLSS workflow's allowlist. COM is initialised on the worker for `ShellExecuteW`. On Linux, desktop programs are started with only their three standard files (close-on-exec clipboard pipe, `addclosefrom_np`). The zip is written to a fresh `.new` file (`O_EXCL|O_NOFOLLOW` / `CREATE_NEW`) and moved over the old one. `gpu/vk/vk_draw.cpp` `state_list_snapshot()` reads the list under the lock the recorder writes it with, so a record being written at that moment is never half in the zip. - `gpu/vk/vk_main.cpp` logs `vk: driver: <name> <info>` once at start, so the builders can report the driver too (older logs: decoded from the version number). - `nx2/pipeline_share.py`: the same zip for the builders (the list read once, so the bytes checked are the bytes zipped), from the game folder (version from the last log or `hookshot_version.txt`, GPU and driver from the last log, settings from `hookshot_options.ini`, mods from `mods/`). The Linux builder imports it; the Windows builder runs it with `--json` with its own Python, then does the clipboard, Explorer and browser itself. It also works by hand: `python -I nx2/pipeline_share.py <game folder>`. - Docs: readme (new "Send your pipeline list" section under "Want to help?", the menu row, the stutter troubleshooting row), `assets/pipeline_states/README.md`, `docs/development.md`. ## Tests `tests/test_pipeline_share.py` (game-free, picked up by `run_all.py`): - the zip from both the C++ header and the Python holds exactly `hookshot_pipeline_states.bin` (byte-identical) and `hookshot_pipeline_report.txt`, and nothing else from a game folder that also has saves, caches, the options file and the log; the CRCs check; - the template has every field in the issue's order, the expected values from a synthetic options file, log and mods folder, "where you played" empty, and no path of the player's; - missing, not-a-list, cut-short and empty lists are refused with the reason and no zip is written; C++ and Python agree on every case; - the C++ and Python field lists are the same, and the Python's names for the upscalers, render resolution shares, output sizes, frame generation and HDR values match `vk_menu.cpp` / `upscaler_rows.h`; - the list holds only pipelines the game drew: `state_record()` is reached only from `draw()` through `graphics_pipeline()` and `pipeline_note()`, never from the precompile, predicted pipelines or the shipped list; - the C++ and Python Output resolution lines agree (Handheld profile, the game's own size, Ultrawide); - the menu row saves on Enter (not the Controls page catch-all), checks before zipping, opens nothing under `HOS_HEADLESS`, opens the browser on a worker with COM initialised, gives desktop programs none of the game's files, and uploads nothing; - the Linux builder's action, with stand-ins for the clipboard and `xdg-open`: the zip, then copy, folder, issue; a refusal opens nothing. Full `python -I -B tests/run_all.py` on Linux, on the final tree (main merged in): 401 passed, 0 failed, 12 skipped (no dotnet, PIL or some headers here). ## Checked by hand - **Game (Linux, RTX 3090, NVIDIA 620.70.06, headless run, `HOS_HEADLESS=1 HOS_NO_SDL=1`, title screen; build of this branch with main merged in, v2.1.0-rc.3-120-g5df019f)**: the menu opened with `RX_VK_MENU_SCRIPT`, then Debug tab, Save my pipeline list, Enter. The row changed from Save to Saved, and the help box said where the zip went and what to do next. The log said `menu: Save my pipeline list: hookshot-pipelines-2026-11-11.zip, 672 pipelines, in Downloads`. The zip held exactly `hookshot_pipeline_states.bin` (byte-identical to the game folder's) and `hookshot_pipeline_report.txt`: ``` - Hookshot version: v2.1.0-rc.3-120-g5df019f (source 9d480ad7) - OS, GPU and driver: Linux (CachyOS, kernel 7.3.3-1-cachyos); NVIDIA GeForce RTX 3090, driver NVIDIA 620.70.06 - Output resolution: 2048x1152 (the game's own) - Upscaler + render resolution: Game's own, Auto (dynamic) - Frame generation / HDR: Off / Off - Mods or texture packs: none - Where you played (areas, dungeons, age, night/rain, ...): - Pipelines: 672 (version 2) ``` The run was headless, so as designed it opened nothing. - **Linux builder**: the launcher page in a nested headless sway, the button pressed through Tk: the zip landed in the Downloads folder from `user-dirs.dirs`, `xdg-open` was asked for the folder and the issue, the status line says what happened (screenshots). - **Windows builder**: compiles (`dotnet build -c Debug -p:EnableWindowsTargeting=true -warnaserror`, .NET 8 SDK on Linux, 0 warnings). Not run on Windows: the clipboard, Explorer and browser calls there are untested. - **Windows game**: `vk_menu.cpp` and `vk_main.cpp` compile with llvm-mingw. `vk_menu.o` imports only kernel32, user32 and UCRT functions; shell32 and ole32 are loaded at run time. Not run on Windows. ## Screenshots - In-game menu, Debug tab, the row selected: ![menu-debug-before-save](https://raw.githubusercontent.com/hckrman101/Hookshot/700a25aba571da5ac3b913c8fd3f8fb1f6b4e521/plshare2/menu-debug-before-save.png) - After Enter: the row says Saved, and the help box says where the zip went and what to do: ![menu-debug-after-save](https://raw.githubusercontent.com/hckrman101/Hookshot/700a25aba571da5ac3b913c8fd3f8fb1f6b4e521/plshare2/menu-debug-after-save.png) - Linux builder launcher page after pressing the button: ![builder-linux-after](https://raw.githubusercontent.com/hckrman101/Hookshot/700a25aba571da5ac3b913c8fd3f8fb1f6b4e521/plshare2/builder-linux-after.png) ## Suggested new body for #2034 (not edited) When Hookshot meets a pipeline it hasn't built on your PC yet, it builds it on the spot, which causes a stutter, or a short white wash on the first field load. Releases ship a list of known pipelines that the first start builds ahead of time. The list only covers what someone has played, so **send us the list your game wrote while you played** and we'll merge it into the next release. **Is it safe?** Yes. The file holds no game code, game data or personal info: only shader hashes, formats and draw state. See [`assets/pipeline_states/README.md`](https://git.magicbean.dev/MagicBeanDev/Hookshot/src/branch/main/assets/pipeline_states/README.md). ## How to submit 1. Play on a recent build (a build from **main**), see https://github.com/hckrman101/HookshotDownloads/releases 2. Save your list: - **In the game:** open the menu (Insert or F1), go to the **Debug** tab and choose **Save my pipeline list** (under Developer). - **Or in Hookshot Builder:** on the game folder's launcher page, press **Save my pipeline list**. It saves `hookshot-pipelines-<date>.zip` in your Downloads folder, copies a filled-in report to your clipboard and opens this issue. 3. Comment here: drag the zip in, paste the report, and fill in **where you played**. Nothing is uploaded for you, and the zip holds only `hookshot_pipeline_states.bin` and the report (version, OS, GPU and driver, output and render resolution, upscaler, frame generation, HDR, mods). If your list is missing or damaged, it tells you and saves nothing. Settings matter, so lists from different GPUs, upscalers and resolutions are especially welcome, as are areas few people reach (late dungeons, bosses, night, rain, underwater). <details> <summary>On a build without Save my pipeline list</summary> Zip `hookshot_pipeline_states.bin` from next to the Hookshot executable (only that file: no `hookshot_shaders.bin`, saves or anything else from the game folder), drag the zip into a comment here and fill in: ``` - Hookshot version: - OS, GPU and driver: - Output resolution: - Upscaler + render resolution (game's / DLSS / FSR 3.1 / FSR 4; Quality / Balanced / ...): - Frame generation / HDR: on / off - Mods or texture packs: - Where you played (areas, dungeons, age, night/rain, ...): ``` Or, from a clone of the repository: `python -I nx2/pipeline_share.py <game folder>` makes the same zip and report. </details> ## What happens next We merge each list with `nx2/pipeline_states.py` (each pipeline kept once) and note its source in `assets/pipeline_states/`. Lists that only repeat known pipelines add nothing, and that's fine. Thanks! (Changes from the current body: the one-click steps lead; the old manual steps and template move into a collapsed section; the AI-agent paragraph is dropped because the button does that job; the safety link points at `assets/pipeline_states/README.md`, because the current `assets/pipeline_states.md` link is broken.) ## Not tested - The non-headless path in the game (the worker): clipboard (`wl-copy`/`xclip`/`xsel`; on Windows, `CF_UNICODETEXT`), opening the folder and opening the browser. Test runs are headless by rule, so nothing was opened. - Anything on Windows: the game's menu action, the WPF builder's button (it compiles only), Explorer `/select` and the Downloads known folder. - The Linux AppImage. `xdg-open` inherits the AppImage's environment there, as the game's existing update prompt already does. - The refusal message in the game (missing, empty or damaged list). It is covered by the game-free test for the same C++ code, not by a run. - The Python/Qt Windows builder (`builder/py`) has no button yet. It could call `nx2/pipeline_share.py` the same way.

Joey-Magnus (Migrated from github.com) reviewed

2026-11-11 21:27:04 +00:00

Joey-Magnus (Migrated from github.com) left a comment

The red checks are this PR's own:

test_recorder_sync fails because vk_work.h is no longer the last include in vk_menu.cpp.
The Windows link check fails because hookshot_vk.dll now imports SHELL32 and ole32. Those calls either move to the host, or CI's allowlist widens, which is a maintainer choice.

The red checks are this PR's own: - `test_recorder_sync` fails because `vk_work.h` is no longer the last include in `vk_menu.cpp`. - The Windows link check fails because `hookshot_vk.dll` now imports `SHELL32` and `ole32`. Those calls either move to the host, or CI's allowlist widens, which is a maintainer choice.