MagicBeanDev/Longshot
MagicBeanDev/Longshot
Watch
58
71
Fork
You've already forked Longshot
 74
Code Issues 138 Pull requests 210 Projects Releases 8 Packages Wiki Activity Actions 474

perf(prepare): the segment heap, so the driver's compiles run side by side on Windows #15

Merged
HumanBean merged 2 commits from madebyjordan/Longshot:perf/prepare-segment-heap into main 2026-11-12 23:13:53 +00:00
Conversation 1 Commits 2 Files changed 9 +183 -3
madebyjordan commented 2026-11-12 23:02:10 +00:00
Contributor

Summary

On AMD's Windows driver, a first start's shader preparation takes hours, and giving it more CPU threads made it slower, not faster: the driver's compiler threads all wait on one lock in Windows' default memory heap. This makes the game use Windows' segment heap, which has no such lock, so the preparation's first part can use every CPU. Preparation gets about a third faster; play is unchanged.

Before and after

RX 6700 XT, Ryzen 5 5600, 16 GB, Windows 11, AMD 26.10.1. The same build both ways (one exe with the manifest's segment heap line, a copy without it), from cold caches, --prepare-shaders.

Before: NT heap, 6 workers After: segment heap, 12 workers Change
Shader preparation, first part 1,478 s 982 s -34% (8 min less)
Pipelines predicted in the first 5 minutes 1,115 2,153 +93%
Peak memory during preparation 9.0 GB 12.1 GB +3.1 GB
In play: Hyrule Field, 1440p, FSR 4, four alternated pairs 47.3-51.2 fps 49.2-51.3 fps no change (pairs -1.4% to +4.0%, noise)

Only Windows changes: on Linux, or a Windows that ignores the manifest, the worker count stays as before. Not measured yet: NVIDIA, Intel, Windows 10.

Details

Why

On a Ryzen 5 5600 (6 cores, 12 threads) with an RX 6700 XT, a cold --prepare-shaders run of main spent 1,466 s in its first part (prepare_run: every program translated, and AMD's compile of 9,691 compute variants), most of it in the driver. More workers did not help:

  • 12 workers instead of the usual 6 were slower, using 3.75 CPUs on average.
  • Stack samples of the workers (suspend, walk, attribute by module; 1,500 samples) showed 76% of them waiting, nearly all at amdvlk64.dll -> RtlAllocateHeap / RtlFreeHeap -> RtlEnterCriticalSection -> NtWaitForAlertByThreadId: the NT heap's lock.
  • A standalone harness compiling one of the large compute shaders (217 KB of SPIR-V) with the same device setup: 1 thread 0.48 builds/s, 6 threads 1.43, 12 threads 1.58. Two separate processes of 6 threads each (two heaps) reached 1.95, and one process on the segment heap 2.00, so the loss is the heap and not the CPU. VkAllocationCallbacks did not help: the driver does not use them for its compiler's memory.

What changed

  • nx2/win_manifest.py: the game's manifest asks for <heapType>SegmentHeap</heapType> (Windows 10 2004 and newer; older Windows ignores it).
  • gpu/vk/prepare_workers.h (new), vk_progcache.cpp prepare_cpus(): prepare_run's pool before the game is half_cpus() as before, unless the process heap really is the segment heap (read from the heap's header, so a Windows that ignored the manifest, or Linux, keeps the old count). Then every usable CPU gets a worker, one more for each 768 MB of free physical memory past 4 GB (each worker past half the CPUs took 0.4-0.5 GB more at the peak). The log says which: vk: preparation: 12 workers for the shader programs (12 CPUs usable; the segment heap: ...).
  • The prediction and the precompile already run on every usable CPU; they gain from the heap alone.
  • Unchanged: the preparation in play (RX_VK_PREPARE_BACKGROUND), Linux, and every other pool.

Measured

Windows 11 (26200), Ryzen 5 5600, 16 GB, RX 6700 XT, AMD 26.10.1. Cold runs: the game's caches emptied, and a new folder path each run, which gives the AMD driver an empty cache too (a new path got a new file in %LOCALAPPDATA%\AMD\VkCache each time). --prepare-shaders, no window.

Same build (this branch, v2.1.0-rc.3-299-gdae19300): its own exe against a copy of the same exe whose manifest has the heapType line taken out:

NT heap (6 workers) segment heap (12 workers)
First part (vk: prepared ... in) 1,478 s 982 s (-34%)
Predicted pipelines in its first 300 s 1,115 2,153 (+93%)
Peak private memory 9.0 GB 12.1 GB

Earlier, on main 51da0464 with the manifest patched into a copy of the exe: first part 1,466 s (NT heap, 6 workers) against 904 s (segment heap, 12 workers); predicted pipelines in the first 300 s of a fresh process 1,495 (NT heap) against 2,358 (segment heap, after its first part in the same process).

In play (main's renderer and exe, the same exe with and without the manifest line; the field bench save: Hyrule Field by day, 2560x1440 output with AMD FSR 4, standing, 150 s each, four alternated pairs): fps median, NT heap / segment heap: 50.8 / 50.1, 51.2 / 50.7, 49.5 / 51.3, 47.3 / 49.2 (pair differences -1.4%, -1.0%, +3.6%, +4.0%: run-to-run noise, no direction). GPU time 13.0-13.6 ms and replay draw time 9.7-10.9 ms move with the fps in both; peak private memory 21.1-21.4 GB in both. (Three earlier runs, both kinds among them, were all at 45 fps while the PC was busy, and are left out.)

Not measured: NVIDIA and Intel on Windows (they get the heap, and on the segment heap the same worker count change), a full end-to-end preparation (hours on this PC either way), Windows 10.

Overlap

See NotRexed/Hookshot#2696, NotRexed/Hookshot#2505.

Those are Hookshot pull requests from before the move; whichever of them come to Longshot overlap the same way.

Tests

Carried over from NotRexed/Hookshot#2895, unchanged apart from the rename: rebased onto Longshot's main 7afe6ae1, with (hookshot) comments now (longshot). The measurements above are from that pull request's Windows builds (on Hookshot main dae19300); the change's code is the same, but it has not been measured again on Longshot's main.

  • tests/test_prepare_workers.py (new): compiles prepare_workers.h: the NT heap's count, one worker per 768 MB past the reserve, the usable CPUs and 32 as limits, the heap signatures; and checks prepare_run and the manifest use them.
  • tests/test_utf8_paths.py: the manifest also has heapType SegmentHeap; its probe exe still starts from a folder named in Japanese and emoji with the new manifest (UTF-8 code page as before).
  • tests/test_prepare_cpus.py: prepare_run is sized by prepare_cpus, which starts from half_cpus and usable_cpus.
  • tests/run_all.py on Linux after the rebase (CachyOS, GCC 16.2, Python 3.14): 453 passed, 4 failed, 72 skipped; main 7afe6ae1 alone fails the same 4 (numpy, zstandard and Tk not installed here, and a GCC 16 internal compiler error in test_lower_tld_ms.py), and also test_ship_state_list.py once.
  • Before the move, tests/run_all.py on Windows (llvm-mingw clang, the Vulkan headers from the builder's CPM cache): 469 passed, 11 failed, 41 skipped; the same 11 fail on main dae19300 alone on this PC (builder, portable-archive and readme checks, and test_program_cache_one_variant.py, which does not compile NotRexed/Hookshot#1630's disk_zstd here).

🤖 Generated with Claude Code

## Summary On AMD's Windows driver, a first start's shader preparation takes hours, and giving it more CPU threads made it slower, not faster: the driver's compiler threads all wait on one lock in Windows' default memory heap. This makes the game use Windows' segment heap, which has no such lock, so the preparation's first part can use every CPU. Preparation gets about a third faster; play is unchanged. ## Before and after RX 6700 XT, Ryzen 5 5600, 16 GB, Windows 11, AMD 26.10.1. The same build both ways (one exe with the manifest's segment heap line, a copy without it), from cold caches, `--prepare-shaders`. | | Before: NT heap, 6 workers | After: segment heap, 12 workers | Change | |---|---|---|---| | Shader preparation, first part | 1,478 s | **982 s** | **-34%** (8 min less) | | Pipelines predicted in the first 5 minutes | 1,115 | **2,153** | **+93%** | | Peak memory during preparation | 9.0 GB | 12.1 GB | +3.1 GB | | In play: Hyrule Field, 1440p, FSR 4, four alternated pairs | 47.3-51.2 fps | 49.2-51.3 fps | no change (pairs -1.4% to +4.0%, noise) | Only Windows changes: on Linux, or a Windows that ignores the manifest, the worker count stays as before. Not measured yet: NVIDIA, Intel, Windows 10. ## Details ### Why On a Ryzen 5 5600 (6 cores, 12 threads) with an RX 6700 XT, a cold `--prepare-shaders` run of main spent 1,466 s in its first part (`prepare_run`: every program translated, and AMD's compile of 9,691 compute variants), most of it in the driver. More workers did not help: - 12 workers instead of the usual 6 were **slower**, using 3.75 CPUs on average. - Stack samples of the workers (suspend, walk, attribute by module; 1,500 samples) showed **76% of them waiting**, nearly all at `amdvlk64.dll -> RtlAllocateHeap / RtlFreeHeap -> RtlEnterCriticalSection -> NtWaitForAlertByThreadId`: the NT heap's lock. - A standalone harness compiling one of the large compute shaders (217 KB of SPIR-V) with the same device setup: 1 thread 0.48 builds/s, 6 threads 1.43, 12 threads 1.58. Two separate processes of 6 threads each (two heaps) reached 1.95, and one process on the segment heap 2.00, so the loss is the heap and not the CPU. `VkAllocationCallbacks` did not help: the driver does not use them for its compiler's memory. ### What changed - `nx2/win_manifest.py`: the game's manifest asks for `<heapType>SegmentHeap</heapType>` (Windows 10 2004 and newer; older Windows ignores it). - `gpu/vk/prepare_workers.h` (new), `vk_progcache.cpp` `prepare_cpus()`: `prepare_run`'s pool before the game is `half_cpus()` as before, unless the process heap really is the segment heap (read from the heap's header, so a Windows that ignored the manifest, or Linux, keeps the old count). Then every usable CPU gets a worker, one more for each 768 MB of free physical memory past 4 GB (each worker past half the CPUs took 0.4-0.5 GB more at the peak). The log says which: `vk: preparation: 12 workers for the shader programs (12 CPUs usable; the segment heap: ...)`. - The prediction and the precompile already run on every usable CPU; they gain from the heap alone. - Unchanged: the preparation in play (`RX_VK_PREPARE_BACKGROUND`), Linux, and every other pool. ### Measured Windows 11 (26200), Ryzen 5 5600, 16 GB, RX 6700 XT, AMD 26.10.1. Cold runs: the game's caches emptied, and a new folder path each run, which gives the AMD driver an empty cache too (a new path got a new file in `%LOCALAPPDATA%\AMD\VkCache` each time). `--prepare-shaders`, no window. **Same build** (this branch, `v2.1.0-rc.3-299-gdae19300`): its own exe against a copy of the same exe whose manifest has the `heapType` line taken out: | | NT heap (6 workers) | segment heap (12 workers) | |---|---|---| | First part (`vk: prepared ... in`) | 1,478 s | **982 s (-34%)** | | Predicted pipelines in its first 300 s | 1,115 | **2,153 (+93%)** | | Peak private memory | 9.0 GB | 12.1 GB | Earlier, on main `51da0464` with the manifest patched into a copy of the exe: first part 1,466 s (NT heap, 6 workers) against 904 s (segment heap, 12 workers); predicted pipelines in the first 300 s of a fresh process 1,495 (NT heap) against 2,358 (segment heap, after its first part in the same process). **In play** (main's renderer and exe, the same exe with and without the manifest line; the field bench save: Hyrule Field by day, 2560x1440 output with AMD FSR 4, standing, 150 s each, four alternated pairs): fps median, NT heap / segment heap: 50.8 / 50.1, 51.2 / 50.7, 49.5 / 51.3, 47.3 / 49.2 (pair differences -1.4%, -1.0%, +3.6%, +4.0%: run-to-run noise, no direction). GPU time 13.0-13.6 ms and replay draw time 9.7-10.9 ms move with the fps in both; peak private memory 21.1-21.4 GB in both. (Three earlier runs, both kinds among them, were all at 45 fps while the PC was busy, and are left out.) Not measured: NVIDIA and Intel on Windows (they get the heap, and on the segment heap the same worker count change), a full end-to-end preparation (hours on this PC either way), Windows 10. ### Overlap - [NotRexed/Hookshot#2725](https://github.com/NotRexed/Hookshot/pull/2725) / [NotRexed/Hookshot#2727](https://github.com/NotRexed/Hookshot/pull/2727) / [NotRexed/Hookshot#2728](https://github.com/NotRexed/Hookshot/pull/2728) (worker counts, a RAM guard, a bench script) and [NotRexed/Hookshot#2414](https://github.com/NotRexed/Hookshot/pull/2414) / [NotRexed/Hookshot#1610](https://github.com/NotRexed/Hookshot/pull/1610) edit the same pool sizing in `prepare_run` / `predict_run` / `precompile_run`. On the NT heap, more workers ran slower here, so those counts are worth measuring again on the segment heap. - [NotRexed/Hookshot#2814](https://github.com/NotRexed/Hookshot/pull/2814) (pipeline binaries on AMD Windows) and [NotRexed/Hookshot#2760](https://github.com/NotRexed/Hookshot/pull/2760) (load programs at create) are about other costs; no overlap in code. See [NotRexed/Hookshot#2696](https://github.com/NotRexed/Hookshot/pull/2696), [NotRexed/Hookshot#2505](https://github.com/NotRexed/Hookshot/pull/2505). Those are Hookshot pull requests from before the move; whichever of them come to Longshot overlap the same way. ### Tests Carried over from [NotRexed/Hookshot#2895](https://github.com/NotRexed/Hookshot/pull/2895), unchanged apart from the rename: rebased onto Longshot's main 7afe6ae1, with `(hookshot)` comments now `(longshot)`. The measurements above are from that pull request's Windows builds (on Hookshot main dae19300); the change's code is the same, but it has not been measured again on Longshot's main. - `tests/test_prepare_workers.py` (new): compiles `prepare_workers.h`: the NT heap's count, one worker per 768 MB past the reserve, the usable CPUs and 32 as limits, the heap signatures; and checks `prepare_run` and the manifest use them. - `tests/test_utf8_paths.py`: the manifest also has `heapType SegmentHeap`; its probe exe still starts from a folder named in Japanese and emoji with the new manifest (UTF-8 code page as before). - `tests/test_prepare_cpus.py`: `prepare_run` is sized by `prepare_cpus`, which starts from `half_cpus` and `usable_cpus`. - `tests/run_all.py` on Linux after the rebase (CachyOS, GCC 16.2, Python 3.14): 453 passed, 4 failed, 72 skipped; main 7afe6ae1 alone fails the same 4 (numpy, zstandard and Tk not installed here, and a GCC 16 internal compiler error in `test_lower_tld_ms.py`), and also `test_ship_state_list.py` once. - Before the move, `tests/run_all.py` on Windows (llvm-mingw clang, the Vulkan headers from the builder's CPM cache): 469 passed, 11 failed, 41 skipped; the same 11 fail on main dae19300 alone on this PC (builder, portable-archive and readme checks, and `test_program_cache_one_variant.py`, which does not compile [NotRexed/Hookshot#1630](https://github.com/NotRexed/Hookshot/pull/1630)'s `disk_zstd` here). 🤖 Generated with [Claude Code](https://claude.com/claude-code) 
The shader preparation's first part spent most of its time with its workers waiting on the NT heap's one lock: AMD's
compiler allocates and frees through the process heap from every worker at once (sampled: amdvlk64.dll -> RtlAllocateHeap /
RtlFreeHeap -> RtlEnterCriticalSection, 76% of the workers' samples). The game's manifest now asks for the segment heap
(Windows 10 2004 and newer), and with it prepare_run gives every usable CPU a worker, as far as the free memory allows
(gpu/vk/prepare_workers.h). Without it (Linux, older Windows) the count is half the CPUs, as before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
test(prepare): prepare_run sized by prepare_cpus; tested configurations for the segment heap
Some checks were skipped
Website / Website on Cloudflare (pull_request_target) Has been skipped
Details
c56a452b1a 
Co-Authored-By: Claude Opus 5.5 <[email protected]>
HumanBean approved these changes 2026-11-12 23:09:45 +00:00
HumanBean deleted branch perf/prepare-segment-heap 2026-11-12 23:13:54 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
HumanBean
Labels
Clear labels   
area/accessibility
Barriers for people with disabilities

  
area/audio
Sound and music

  
area/input
Controllers, keyboard, mouse

  
area/install
Installing, updating, building from your game

  
area/menu
In-game menu and settings

  
area/renderer
Vulkan renderer, shaders, effects

  
area/saves
Save files

  
area/upscaling
DLSS, FSR, frame generation

  
good first issue
A good start for a new contributor

  
gpu/amd
AMD GPUs

  
gpu/igpu
Integrated GPUs

  
gpu/intel
Intel GPUs

  
gpu/nvidia
NVIDIA GPUs

  
help wanted
Extra hands or knowledge wanted

  
kind
bug
 Something doesn't work as it should

  
kind
chore
 Build, CI, dependencies, tidying

  
kind
docs
 Documentation

  
kind
feature
 A new feature or an improvement

  
kind
performance
 Frame rate, frame time, loading, memory

  
kind
question
 A question rather than a change

  
needs measurements
Waiting for before/after measurements

  
platform/linux
Affects Linux (a change for every system carries both Windows and Linux)

  
platform/macos
macOS (Apple silicon or Intel Macs)

  
platform/steam-deck
Steam Deck

  
platform/windows
Affects Windows (a change for every system carries both Windows and Linux)

  
preview
Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  
priority
critical
 Crashes, data loss, or blocks most players

  
priority
high
 Important: next in line

  
priority
low
 Nice to have

  
status
blocked
 Waits on something else first

  
status
confirmed
 Reproduced or agreed on

  
status
duplicate
 Already reported or proposed

  
status
invalid
 Not a problem in this project

  
status
needs-info
 Waiting for more information from the reporter

  
status
wontfix
 Will not be worked on

  
testing wanted
Wanted: a test on other hardware (posts a test request in #test-requests)

  
ci: approved
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
ci: approved
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!15
No description provided.