MagicBeanDev/Hookshot

Archived

Watch

Fork 0

perf(prepare): the segment heap, so the driver's compiles run side by side on Windows #2895

Open

github-sync wants to merge 2 commits from github/pr/2895 into main

github-sync commented

2026-11-12 19:01:05 +00:00

Owner

@madebyjordan opened this pull request on GitHub · 2026-11-12 18:51 UTC · original

A first start (or the Builder's shader preparation) on AMD's Windows driver spends hours compiling, and adding CPU threads did not make it faster: the driver's compiler threads queue on one lock in Windows' default process heap. This switches the game to Windows' segment heap and, when it is in use, gives the preparation's first part a worker on every CPU.

Why

On a Ryzen 5 5600 (6 cores, 12 threads) with an RX 6700 XT, a cold --prepare-shaders run of main spent 1,466 s in its first part (prepare_run: every program translated, and AMD's compile of 9,691 compute variants), most of it in the driver. More workers did not help:

12 workers instead of the usual 6 were slower, using 3.75 CPUs on average.
Stack samples of the workers (suspend, walk, attribute by module; 1,500 samples) showed 76% of them waiting, nearly all at amdvlk64.dll -> RtlAllocateHeap / RtlFreeHeap -> RtlEnterCriticalSection -> NtWaitForAlertByThreadId: the NT heap's lock.
A standalone harness compiling one of the large compute shaders (217 KB of SPIR-V) with the same device setup: 1 thread 0.48 builds/s, 6 threads 1.43, 12 threads 1.58. Two separate processes of 6 threads each (two heaps) reached 1.95, and one process on the segment heap 2.00, so the loss is the heap and not the CPU. VkAllocationCallbacks did not help: the driver does not use them for its compiler's memory.

What changed

nx2/win_manifest.py: the game's manifest asks for <heapType>SegmentHeap</heapType> (Windows 10 2004 and newer; older Windows ignores it).
gpu/vk/prepare_workers.h (new), vk_progcache.cpp prepare_cpus(): prepare_run's pool before the game is half_cpus() as before, unless the process heap really is the segment heap (read from the heap's header, so a Windows that ignored the manifest, or Linux, keeps the old count). Then every usable CPU gets a worker, one more for each 768 MB of free physical memory past 4 GB (each worker past half the CPUs took 0.4-0.5 GB more at the peak). The log says which: vk: preparation: 12 workers for the shader programs (12 CPUs usable; the segment heap: ...).
The prediction and the precompile already run on every usable CPU; they gain from the heap alone.
Unchanged: the preparation in play (RX_VK_PREPARE_BACKGROUND), Linux, and every other pool.

Measured

Windows 11 (26200), Ryzen 5 5600, 16 GB, RX 6700 XT, AMD 26.10.1. Cold runs: empty Hookshot caches, and a new folder path each run, which gives the AMD driver an empty cache too (a new path got a new file in %LOCALAPPDATA%\AMD\VkCache each time). --prepare-shaders, no window.

Same build (this branch, v2.1.0-rc.3-299-gdae19300): its own exe against a copy of the same exe whose manifest has the heapType line taken out:

	NT heap (6 workers)	segment heap (12 workers)
First part (`vk: prepared ... in`)	1,478 s	982 s (-34%)
Predicted pipelines in its first 300 s	1,115	2,153 (+93%)
Peak private memory	9.0 GB	12.1 GB

Earlier, on main 51da0464 with the manifest patched into a copy of the exe: first part 1,466 s (NT heap, 6 workers) against 904 s (segment heap, 12 workers); predicted pipelines in the first 300 s of a fresh process 1,495 (NT heap) against 2,358 (segment heap, after its first part in the same process).

In play (main's renderer and exe, the same exe with and without the manifest line; the field bench save: Hyrule Field by day, 2560x1440 output with AMD FSR 4, standing, 150 s each, four alternated pairs): fps median, NT heap / segment heap: 50.8 / 50.1, 51.2 / 50.7, 49.5 / 51.3, 47.3 / 49.2 (pair differences -1.4%, -1.0%, +3.6%, +4.0%: run-to-run noise, no direction). GPU time 13.0-13.6 ms and replay draw time 9.7-10.9 ms move with the fps in both; peak private memory 21.1-21.4 GB in both. (Three earlier runs, both kinds among them, were all at 45 fps while the PC was busy, and are left out.)

Not measured: NVIDIA and Intel on Windows (they get the heap, and on the segment heap the same worker count change), a full end-to-end preparation (hours on this PC either way), Windows 10.

Overlap

#2725 / #2727 / #2728 (worker counts, a RAM guard, a bench script) and #2414 / #1610 edit the same pool sizing in prepare_run / predict_run / precompile_run. On the NT heap, more workers ran slower here, so those counts are worth measuring again on the segment heap.
#2814 (pipeline binaries on AMD Windows) and #2760 (load programs at create) are about other costs; no overlap in code.

See #2696, #2505.

Tests

tests/test_prepare_workers.py (new): compiles prepare_workers.h: the NT heap's count, one worker per 768 MB past the reserve, the usable CPUs and 32 as limits, the heap signatures; and checks prepare_run and the manifest use them.
tests/test_utf8_paths.py: the manifest also has heapType SegmentHeap; its probe exe still starts from a folder named in Japanese and emoji with the new manifest (UTF-8 code page as before).
tests/test_prepare_cpus.py: prepare_run is sized by prepare_cpus, which starts from half_cpus and usable_cpus.
tests/run_all.py on Windows (llvm-mingw clang, the Vulkan headers from the builder's CPM cache): 469 passed, 11 failed, 41 skipped; the same 11 fail on main dae19300 alone on this PC (builder, portable-archive and readme checks, and test_program_cache_one_variant.py, which does not compile #1630's disk_zstd here).

🤖 Generated with Claude Code

> [@madebyjordan](https://github.com/madebyjordan) opened this pull request on GitHub · 2026-11-12 18:51 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2895) A first start (or the Builder's shader preparation) on AMD's Windows driver spends hours compiling, and adding CPU threads did not make it faster: the driver's compiler threads queue on one lock in Windows' default process heap. This switches the game to Windows' segment heap and, when it is in use, gives the preparation's first part a worker on every CPU. ## Why On a Ryzen 5 5600 (6 cores, 12 threads) with an RX 6700 XT, a cold `--prepare-shaders` run of main spent 1,466 s in its first part (`prepare_run`: every program translated, and AMD's compile of 9,691 compute variants), most of it in the driver. More workers did not help: - 12 workers instead of the usual 6 were **slower**, using 3.75 CPUs on average. - Stack samples of the workers (suspend, walk, attribute by module; 1,500 samples) showed **76% of them waiting**, nearly all at `amdvlk64.dll -> RtlAllocateHeap / RtlFreeHeap -> RtlEnterCriticalSection -> NtWaitForAlertByThreadId`: the NT heap's lock. - A standalone harness compiling one of the large compute shaders (217 KB of SPIR-V) with the same device setup: 1 thread 0.48 builds/s, 6 threads 1.43, 12 threads 1.58. Two separate processes of 6 threads each (two heaps) reached 1.95, and one process on the segment heap 2.00, so the loss is the heap and not the CPU. `VkAllocationCallbacks` did not help: the driver does not use them for its compiler's memory. ## What changed - `nx2/win_manifest.py`: the game's manifest asks for `<heapType>SegmentHeap</heapType>` (Windows 10 2004 and newer; older Windows ignores it). - `gpu/vk/prepare_workers.h` (new), `vk_progcache.cpp` `prepare_cpus()`: `prepare_run`'s pool before the game is `half_cpus()` as before, unless the process heap really is the segment heap (read from the heap's header, so a Windows that ignored the manifest, or Linux, keeps the old count). Then every usable CPU gets a worker, one more for each 768 MB of free physical memory past 4 GB (each worker past half the CPUs took 0.4-0.5 GB more at the peak). The log says which: `vk: preparation: 12 workers for the shader programs (12 CPUs usable; the segment heap: ...)`. - The prediction and the precompile already run on every usable CPU; they gain from the heap alone. - Unchanged: the preparation in play (`RX_VK_PREPARE_BACKGROUND`), Linux, and every other pool. ## Measured Windows 11 (26200), Ryzen 5 5600, 16 GB, RX 6700 XT, AMD 26.10.1. Cold runs: empty Hookshot caches, and a new folder path each run, which gives the AMD driver an empty cache too (a new path got a new file in `%LOCALAPPDATA%\AMD\VkCache` each time). `--prepare-shaders`, no window. **Same build** (this branch, `v2.1.0-rc.3-299-gdae19300`): its own exe against a copy of the same exe whose manifest has the `heapType` line taken out: | | NT heap (6 workers) | segment heap (12 workers) | |---|---|---| | First part (`vk: prepared ... in`) | 1,478 s | **982 s (-34%)** | | Predicted pipelines in its first 300 s | 1,115 | **2,153 (+93%)** | | Peak private memory | 9.0 GB | 12.1 GB | Earlier, on main `51da0464` with the manifest patched into a copy of the exe: first part 1,466 s (NT heap, 6 workers) against 904 s (segment heap, 12 workers); predicted pipelines in the first 300 s of a fresh process 1,495 (NT heap) against 2,358 (segment heap, after its first part in the same process). **In play** (main's renderer and exe, the same exe with and without the manifest line; the field bench save: Hyrule Field by day, 2560x1440 output with AMD FSR 4, standing, 150 s each, four alternated pairs): fps median, NT heap / segment heap: 50.8 / 50.1, 51.2 / 50.7, 49.5 / 51.3, 47.3 / 49.2 (pair differences -1.4%, -1.0%, +3.6%, +4.0%: run-to-run noise, no direction). GPU time 13.0-13.6 ms and replay draw time 9.7-10.9 ms move with the fps in both; peak private memory 21.1-21.4 GB in both. (Three earlier runs, both kinds among them, were all at 45 fps while the PC was busy, and are left out.) Not measured: NVIDIA and Intel on Windows (they get the heap, and on the segment heap the same worker count change), a full end-to-end preparation (hours on this PC either way), Windows 10. ## Overlap - #2725 / #2727 / #2728 (worker counts, a RAM guard, a bench script) and #2414 / #1610 edit the same pool sizing in `prepare_run` / `predict_run` / `precompile_run`. On the NT heap, more workers ran slower here, so those counts are worth measuring again on the segment heap. - #2814 (pipeline binaries on AMD Windows) and #2760 (load programs at create) are about other costs; no overlap in code. See #2696, #2505. ## Tests - `tests/test_prepare_workers.py` (new): compiles `prepare_workers.h`: the NT heap's count, one worker per 768 MB past the reserve, the usable CPUs and 32 as limits, the heap signatures; and checks `prepare_run` and the manifest use them. - `tests/test_utf8_paths.py`: the manifest also has `heapType SegmentHeap`; its probe exe still starts from a folder named in Japanese and emoji with the new manifest (UTF-8 code page as before). - `tests/test_prepare_cpus.py`: `prepare_run` is sized by `prepare_cpus`, which starts from `half_cpus` and `usable_cpus`. - `tests/run_all.py` on Windows (llvm-mingw clang, the Vulkan headers from the builder's CPM cache): 469 passed, 11 failed, 41 skipped; the same 11 fail on main dae19300 alone on this PC (builder, portable-archive and readme checks, and `test_program_cache_one_variant.py`, which does not compile #1630's `disk_zstd` here). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

github-sync added 2 commits

2026-11-12 19:01:05 +00:00

perf(prepare): the segment heap, so the driver's compiles run side by side on Windows abacc8ea27

The shader preparation's first part spent most of its time with its workers waiting on the NT heap's one lock: AMD's
compiler allocates and frees through the process heap from every worker at once (sampled: amdvlk64.dll -> RtlAllocateHeap /
RtlFreeHeap -> RtlEnterCriticalSection, 76% of the workers' samples). The game's manifest now asks for the segment heap
(Windows 10 2004 and newer), and with it prepare_run gives every usable CPU a worker, as far as the free memory allows
(gpu/vk/prepare_workers.h). Without it (Linux, older Windows) the count is half the CPUs, as before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

test(prepare): prepare_run sized by prepare_cpus; tested configurations for the segment heap 46f8e70fa3

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Commenting is not possible because the repository is archived.

No reviewers

No labels

No milestone

No assignees

1 participant

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Hookshot!2895

No description provided.

Rows
Columns