Start faster: an index for the program cache, and the precompile's programs read in parallel #859

Merged

hckrman101 merged 2 commits from boot-program-index into main

2026-11-09 04:59:40 +00:00

hckrman101 commented

2026-11-08 23:22:55 +00:00

(Migrated from github.com)

What

Two changes to how hookshot_programs.bin is opened at every start. Nothing the game does changes: the same programs are loaded and the same pipelines are built before the game starts, in the same order.

An index beside the cache, hookshot_programs.idx. The cache's records follow one another with nothing pointing to them, so finding them meant reading each record's 20-byte head. That is tens of thousands of small reads spread over a file of 4 to 6 GB, one after another, at every start. The index is those heads in file order, 20 bytes each, under a header with the cache's stamp. It is under 1 MB and read in one go.
- A record is added to the index right after it is written to the cache, so the index can fall behind the cache but never get ahead of it.
- At start the index is used only if its stamp matches, its sizes fit inside the cache file, and the last record it names matches that record's head in the cache. Records past the end of the index (written by a build without it, or by a run that stopped between the two writes) are read the old way and added.
- In any other case (index missing, cut short, or not matching) the cache is read the old way and the index is written again. Each record's head is also checked against the index when the record is loaded; a record that doesn't match is translated again, as a damaged record is now.
The precompile's programs are read on several threads. program_cache_load_some read every compute program and every program in the pipeline notes (a few thousand records, a few hundred MB) one at a time, holding shader_mu. Now up to 16 threads read and parse the records, each with its own handle on the file. vkCreateShaderModule is thread-safe. The results go into the program table afterwards, under shader_mu, in the same order as before.

A third commit adds to the precompile's log line how much of its time went to opening the caches and reading the notes and programs.

Why

On a warm start (every cache already built), the first frame came 12.8 s after launch here. 8 s of that went to finding the records in the program cache, before a single pipeline was built. The game thread waits at its first submit for all of it.

On a file system that compresses (btrfs compress=zstd here), each of those 20-byte reads also unpacks a 128 KB block. NTFS without compression has the same problem on a cold disk: one random read per record.

Measurements

RTX 3090 (driver 615.78), Ryzen 9 7900X, Linux, NVMe (btrfs on LUKS, zstd). Program cache with 39,933 programs (6.1 GB), 700 noted graphics pipelines and 3,842 compute pipelines. Same game build, same caches, only hookshot_vk.so swapped. Times are seconds from process start, median of 4 starts each, run alternately A/B, with nothing else running. "Cold" means the folder's files were dropped from the page cache (POSIX_FADV_DONTNEED) before each start. "Warm" starts back to back. With the PC's memory mostly in use, little of a 6 GB file stays cached either way, which is why the two columns are close.

Phase	Before, warm	After, warm	Before, cold	After, cold
exe, image.bin, game init to Vulkan ready	0.3	0.3	0.3	0.4
pipeline cache + finding the program cache's records	8.1	0.2	8.0	0.2
programs read + pipelines built	2.2	1.6	2.2	1.6
renderer up to the first frame (the game's own start)	2.0	2.0	2.1	2.0
first frame	12.8	4.1	12.6	4.2
first frame to the title screen	3.3	3.3	2.7	3.1
title screen to playing (A presses, the save loading)	10.0	9.9	10.3	10.2
playing	25.8	17.3	25.6	17.5

Each start's first frame, warm: before 12.99 / 12.91 / 12.70 / 12.09 s, after 4.34 / 4.12 / 4.01 / 4.03 s.

The disk reads by the time the game starts went from about 1.8 GB to 0.8 GB (/proc/<pid>/io read_bytes).

The old scan grows with the cache, since every program the game translates in play adds a record. On a 4 GB cache of 22,000 programs, a script doing the same reads took 4 to 6 s here.

Tests: tests/test_program_cache_index.py checks the source: the index entry is written after its record, the start-up checks, the head check on load, and the parallel reads joining before the program table is touched. tests/test_program_cache_reader.py now follows the reader into its new function. Index checks, each a real start:

index cut short by 50 bytes: program cache index: 40044 records from it, 3 more read from the cache, then rewritten whole;
last entry's key changed: index ignored, full scan, rewritten;
index deleted: full scan, written again;
the start after each: index whole, nothing scanned.

Notes

vk_shader.cpp is part of HOOKSHOT_SHADER_STAMP, so this change starts the program cache over once, like any translator change: the first start after the update prepares the shaders again (about 70 s here). The cache's format is unchanged, and a build without the index just ignores the .idx file.
No new switch. RX_VK_NO_PROGRAM_CACHE=1 still turns the whole cache off, index included.
Works alongside #700 / #702: they change precompile_run and the draw-time builds, this changes what happens before the first pipeline is built. With #702, the start gets shorter at both ends: before the builds begin (here) and the slow stragglers at the end (there). The only overlap is the log commit, which edits the precompile's final log line that #702 also rewrites. It's a one-line merge, and it can be dropped.
What's left before the first frame: about 1.6 s of reading programs and building pipelines (from the driver's cache), and about 2 s of the game's own start-up after the renderer is up (its main thread busy on the CPU). Building pipelines while the programs are still being read would save a few hundred ms more. That is in precompile_run, which #702 rewrites, so it is left for after.

### What Two changes to how `hookshot_programs.bin` is opened at every start. Nothing the game does changes: the same programs are loaded and the same pipelines are built before the game starts, in the same order. 1. **An index beside the cache, `hookshot_programs.idx`.** The cache's records follow one another with nothing pointing to them, so finding them meant reading each record's 20-byte head. That is tens of thousands of small reads spread over a file of 4 to 6 GB, one after another, at every start. The index is those heads in file order, 20 bytes each, under a header with the cache's stamp. It is under 1 MB and read in one go. - A record is added to the index right after it is written to the cache, so the index can fall behind the cache but never get ahead of it. - At start the index is used only if its stamp matches, its sizes fit inside the cache file, and the last record it names matches that record's head in the cache. Records past the end of the index (written by a build without it, or by a run that stopped between the two writes) are read the old way and added. - In any other case (index missing, cut short, or not matching) the cache is read the old way and the index is written again. Each record's head is also checked against the index when the record is loaded; a record that doesn't match is translated again, as a damaged record is now. 2. **The precompile's programs are read on several threads.** `program_cache_load_some` read every compute program and every program in the pipeline notes (a few thousand records, a few hundred MB) one at a time, holding `shader_mu`. Now up to 16 threads read and parse the records, each with its own handle on the file. `vkCreateShaderModule` is thread-safe. The results go into the program table afterwards, under `shader_mu`, in the same order as before. A third commit adds to the precompile's log line how much of its time went to opening the caches and reading the notes and programs. ### Why On a warm start (every cache already built), the first frame came 12.8 s after launch here. 8 s of that went to finding the records in the program cache, before a single pipeline was built. The game thread waits at its first submit for all of it. On a file system that compresses (btrfs `compress=zstd` here), each of those 20-byte reads also unpacks a 128 KB block. NTFS without compression has the same problem on a cold disk: one random read per record. ### Measurements RTX 3090 (driver 615.78), Ryzen 9 7900X, Linux, NVMe (btrfs on LUKS, zstd). Program cache with 39,933 programs (6.1 GB), 700 noted graphics pipelines and 3,842 compute pipelines. Same game build, same caches, only `hookshot_vk.so` swapped. Times are seconds from process start, median of 4 starts each, run alternately A/B, with nothing else running. "Cold" means the folder's files were dropped from the page cache (`POSIX_FADV_DONTNEED`) before each start. "Warm" starts back to back. With the PC's memory mostly in use, little of a 6 GB file stays cached either way, which is why the two columns are close. | Phase | Before, warm | After, warm | Before, cold | After, cold | |---|---|---|---|---| | exe, image.bin, game init to Vulkan ready | 0.3 | 0.3 | 0.3 | 0.4 | | pipeline cache + **finding the program cache's records** | **8.1** | **0.2** | **8.0** | **0.2** | | programs read + pipelines built | 2.2 | 1.6 | 2.2 | 1.6 | | renderer up to the first frame (the game's own start) | 2.0 | 2.0 | 2.1 | 2.0 | | **first frame** | **12.8** | **4.1** | **12.6** | **4.2** | | first frame to the title screen | 3.3 | 3.3 | 2.7 | 3.1 | | title screen to playing (A presses, the save loading) | 10.0 | 9.9 | 10.3 | 10.2 | | **playing** | **25.8** | **17.3** | **25.6** | **17.5** | Each start's first frame, warm: before 12.99 / 12.91 / 12.70 / 12.09 s, after 4.34 / 4.12 / 4.01 / 4.03 s. The disk reads by the time the game starts went from about 1.8 GB to 0.8 GB (`/proc/<pid>/io` `read_bytes`). The old scan grows with the cache, since every program the game translates in play adds a record. On a 4 GB cache of 22,000 programs, a script doing the same reads took 4 to 6 s here. Tests: `tests/test_program_cache_index.py` checks the source: the index entry is written after its record, the start-up checks, the head check on load, and the parallel reads joining before the program table is touched. `tests/test_program_cache_reader.py` now follows the reader into its new function. Index checks, each a real start: - index cut short by 50 bytes: `program cache index: 40044 records from it, 3 more read from the cache`, then rewritten whole; - last entry's key changed: index ignored, full scan, rewritten; - index deleted: full scan, written again; - the start after each: index whole, nothing scanned. ### Notes - `vk_shader.cpp` is part of `HOOKSHOT_SHADER_STAMP`, so this change starts the program cache over once, like any translator change: the first start after the update prepares the shaders again (about 70 s here). The cache's format is unchanged, and a build without the index just ignores the `.idx` file. - No new switch. `RX_VK_NO_PROGRAM_CACHE=1` still turns the whole cache off, index included. - Works alongside #700 / #702: they change `precompile_run` and the draw-time builds, this changes what happens before the first pipeline is built. With #702, the start gets shorter at both ends: before the builds begin (here) and the slow stragglers at the end (there). The only overlap is the log commit, which edits the precompile's final log line that #702 also rewrites. It's a one-line merge, and it can be dropped. - What's left before the first frame: about 1.6 s of reading programs and building pipelines (from the driver's cache), and about 2 s of the game's own start-up after the renderer is up (its main thread busy on the CPU). Building pipelines while the programs are still being read would save a few hundred ms more. That is in `precompile_run`, which #702 rewrites, so it is left for after.

Commenting is not possible because the repository is archived.

No reviewers