perf(prepare): env override for prepare worker counts #2725

Open

github-sync wants to merge 2 commits from github/pr/2725 into main

github-sync commented

2026-11-12 09:31:58 +00:00

Owner

@yeetbot90 opened this pull request on GitHub · 2026-11-12 09:26 UTC · original

Summary

Centralize prepare pool sizing in prepare_pool_workers (translate / pipelines / background) so defaults stay the old half-usable / all-usable / small-background sizes.
Honour HOOKSHOT_SHADER_WORKERS or RX_VK_PREPARE_WORKERS=<n> (1..64) for A/B when RAM is tight during prepare; no gameplay change when unset.
Log worker count reason plus a one-line CPU/RAM snapshot (prepare_resources_log) at prepare start; document the switches in docs/development.md.

Test plan

python -I -B tests/test_prepare_pool_workers.py / test_prepare_cpus.py (local llvm-mingw)
CI: Python source checks green on Windows + Ubuntu (all four jobs).
Rebuild tip hookshot_vk.dll (RX_VK_ONLY=1) into D:\Hookshot_OOT
AMD RX 9070 XT / Windows --prepare-shaders with HOOKSHOT_SHADER_WORKERS=8:
- noted pipelines: building 28275 on 8 workers (HOOKSHOT_SHADER_WORKERS, usable 24)
- state list also on 8 workers (HOOKSHOT_SHADER_WORKERS)
- warm precompile finished in ~12.4 s (driver VkCache already ~3.4 GB)
Without override, auto path still works; with RAM guard forced to 2 workers (#2727) the override still wins when set

Made with Cursor

> [@yeetbot90](https://github.com/yeetbot90) opened this pull request on GitHub · 2026-11-12 09:26 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2725) ## Summary - Centralize prepare pool sizing in `prepare_pool_workers` (translate / pipelines / background) so defaults stay the old half-usable / all-usable / small-background sizes. - Honour `HOOKSHOT_SHADER_WORKERS` or `RX_VK_PREPARE_WORKERS=<n>` (1..64) for A/B when RAM is tight during prepare; no gameplay change when unset. - Log worker count reason plus a one-line CPU/RAM snapshot (`prepare_resources_log`) at prepare start; document the switches in `docs/development.md`. ## Test plan - [x] `python -I -B tests/test_prepare_pool_workers.py` / `test_prepare_cpus.py` (local llvm-mingw) - [x] CI: Python source checks green on Windows + Ubuntu (all four jobs). - [x] Rebuild tip `hookshot_vk.dll` (`RX_VK_ONLY=1`) into `D:\Hookshot_OOT` - [x] AMD RX 9070 XT / Windows `--prepare-shaders` with `HOOKSHOT_SHADER_WORKERS=8`: - `noted pipelines: building 28275 on 8 workers (HOOKSHOT_SHADER_WORKERS, usable 24)` - state list also `on 8 workers (HOOKSHOT_SHADER_WORKERS)` - warm precompile finished in ~12.4 s (driver VkCache already ~3.4 GB) - [x] Without override, auto path still works; with RAM guard forced to 2 workers (#2727) the override still wins when set Made with [Cursor](https://cursor.com)

github-sync added 1 commit

2026-11-12 09:31:58 +00:00

perf(prepare): env override for prepare worker counts f29faf6382

Centralize pool sizing in prepare_pool_workers so HOOKSHOT_SHADER_WORKERS /
RX_VK_PREPARE_WORKERS can A/B fewer threads when RAM is tight, without changing
the default half/usable/background sizes. Log a one-line CPU and RAM snapshot
at prepare start.

Co-authored-by: Cursor <[email protected]>

github-sync referenced this pull request

2026-11-12 09:31:59 +00:00

perf(prepare): opt-in RAM guard for prepare worker counts #2727

github-sync referenced this pull request

2026-11-12 10:34:10 +00:00

perf(prepare): workers bench script and process WS in prepare log #2728

github-sync added 1 commit

2026-11-12 10:34:11 +00:00

fix(tests): stub prepare pool helpers in stall and CRT-safe pool check 7e0c9fb651

test_prepare_stall's predict harness needed PreparePool / prepare_pool_workers /
prepare_resources_log after the override wiring. The pool-workers check compiles
with _CRT_SECURE_NO_WARNINGS so getenv is not an error under MSVC headers.

Co-authored-by: Cursor <[email protected]>

github-sync commented

2026-11-12 14:44:42 +00:00

Author

Owner

@Joey-Magnus reviewed on GitHub · 2026-11-12 14:34 UTC · original

This commit (f29faf63) is the first commit of #2727, #2728 and #2729. Each of those carries it unchanged plus its own rewrite of the "stub prepare pool helpers" test fix, so the four branches can't be merged one after another without conflicts in tests/test_prepare_pool_workers.py. I'd keep the worker-count work in one PR and close this one, or force-push the full series here and close the others. My review of the combined code is on #2729, and it applies to this commit too: in particular, HOOKSHOT_SHADER_WORKERS also resizes the in-play background pool (vk_progcache.cpp:770, before the PreparePool::Background branch at :778). So the "no gameplay change when unset/set" claim in the docs doesn't hold. And the env values are read with getenv/atoi rather than env_int.

I ran test_prepare_pool_workers.py, test_prepare_cpus.py and test_prepare_stall.py on the #2729 head merged into current main, and they pass.

> [@Joey-Magnus](https://github.com/Joey-Magnus) reviewed on GitHub · 2026-11-12 14:34 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2725#pullrequestreview-5876240341) This commit (f29faf63) is the first commit of #2727, #2728 and #2729. Each of those carries it unchanged plus its own rewrite of the "stub prepare pool helpers" test fix, so the four branches can't be merged one after another without conflicts in tests/test_prepare_pool_workers.py. I'd keep the worker-count work in one PR and close this one, or force-push the full series here and close the others. My review of the combined code is on #2729, and it applies to this commit too: in particular, `HOOKSHOT_SHADER_WORKERS` also resizes the in-play background pool (vk_progcache.cpp:770, before the `PreparePool::Background` branch at :778). So the "no gameplay change when unset/set" claim in the docs doesn't hold. And the env values are read with getenv/atoi rather than `env_int`. I ran test_prepare_pool_workers.py, test_prepare_cpus.py and test_prepare_stall.py on the #2729 head merged into current main, and they pass.

github-sync referenced this pull request

2026-11-12 14:44:44 +00:00

perf(prepare): opt-in RAM guard for prepare worker counts #2727

github-sync referenced this pull request

2026-11-12 14:44:48 +00:00

perf(vk): log GPL availability for stage-library A/B #2729

github-sync referenced this pull request

2026-11-12 19:01:05 +00:00

perf(prepare): the segment heap, so the driver's compiles run side by side on Windows #2895

Commenting is not possible because the repository is archived.

No reviewers