PGO hot list: the game's Zstd decoder at the hot flags whatever the profile says #909

Merged

klimovich008 merged 1 commit from perf-zstd-hot-set into main

2026-11-09 05:31:57 +00:00

klimovich008 commented

2026-11-09 01:11:10 +00:00

(Migrated from github.com)

What

scripts/hotlist.py now always adds the game's Zstd decoder to the hot list, after the functions the profile picks. That's the 22 functions of the decode tree under f_80115c80, called from the job loop's f_80115bf0. They then build in the hot chunks (-O2, with the PGO profile) instead of the cold -O1 chunks.

The shipped work/hot_funcs.txt comes from the release tools, not from git. So this takes effect when the list is next regenerated, or when these 22 names are added to the shipped list directly.

Why

The decoder runs while areas load and stream. A sampled play profile seldom catches it, so it never made the hot set. Stutter on entering an area, and load times, partly wait on it.

I measured each decoder function on its own (harness below). Compiled the way the build compiles today vs. as hot code, median of 5 batches:

	today (-O1)	hot (-O2)
whole decode, 10 frames / 1528 KiB	5516 µs	4605 µs (−16.5%)
single-thread throughput	271 MB/s	324 MB/s
per MiB decoded	3.70 ms	3.09 ms

Repeatable: two independent runs, with and without the PGO profile, agree within 0.1%.
Per function: −8% to −27%; the biggest change is f_801117a0 at −27%.
Size: the 22 functions are 142 KB at -O2 vs 145 KB at -O1. That's +0.3% hot code and a 3 KB smaller exe. hot_order.txt is unchanged, so today's hot code keeps its layout.
Who gains: it shortens loads and streaming stalls. Steady-state fps doesn't change.

Testing

Harness: the lifted C of these functions, built in two files the way the build splits them. It ran 10 whole decodes of zstd frames I generated myself (no game data), four of them with a dictionary loaded through the game's own dictionary setup. It also ran each decoder function from the exact state it was entered with during those decodes.
Correctness: Unicorn decodes every frame back to the original bytes. The native code matches Unicorn on every register and every written page, including the decoded output, at both opt levels: 182 cases, 0 differences.
Whole-game check: test_insn (44,172 cases) and test_leaf (2,091 functions plus 978 call trees, --calls) pass with this list and two other lifter changes applied to v2.0.3.
Game A/B: in the same batch build against stock v2.0.3, median of 3 runs each with equally warm pipeline caches, CPU per frame went down (Hyrule rotating 69.7 → 68.0 ms) and nothing regressed. That scene doesn't measure loading, so it doesn't isolate this change.

Not done: a timed area-load comparison in game.

🤖 Generated with Claude Code

## What `scripts/hotlist.py` now always adds the game's Zstd decoder to the hot list, after the functions the profile picks. That's the 22 functions of the decode tree under `f_80115c80`, called from the job loop's `f_80115bf0`. They then build in the hot chunks (-O2, with the PGO profile) instead of the cold -O1 chunks. The shipped `work/hot_funcs.txt` comes from the release tools, not from git. So this takes effect when the list is next regenerated, or when these 22 names are added to the shipped list directly. ## Why The decoder runs while areas load and stream. A sampled play profile seldom catches it, so it never made the hot set. Stutter on entering an area, and load times, partly wait on it. I measured each decoder function on its own (harness below). Compiled the way the build compiles today vs. as hot code, median of 5 batches: | | today (-O1) | hot (-O2) | |---|---|---| | whole decode, 10 frames / 1528 KiB | 5516 µs | 4605 µs (**−16.5%**) | | single-thread throughput | 271 MB/s | 324 MB/s | | per MiB decoded | 3.70 ms | 3.09 ms | - **Repeatable:** two independent runs, with and without the PGO profile, agree within 0.1%. - **Per function:** −8% to −27%; the biggest change is `f_801117a0` at −27%. - **Size:** the 22 functions are 142 KB at -O2 vs 145 KB at -O1. That's +0.3% hot code and a 3 KB smaller exe. `hot_order.txt` is unchanged, so today's hot code keeps its layout. - **Who gains:** it shortens loads and streaming stalls. Steady-state fps doesn't change. ## Testing - **Harness:** the lifted C of these functions, built in two files the way the build splits them. It ran 10 whole decodes of zstd frames I generated myself (no game data), four of them with a dictionary loaded through the game's own dictionary setup. It also ran each decoder function from the exact state it was entered with during those decodes. - **Correctness:** Unicorn decodes every frame back to the original bytes. The native code matches Unicorn on every register and every written page, including the decoded output, at both opt levels: 182 cases, 0 differences. - **Whole-game check:** `test_insn` (44,172 cases) and `test_leaf` (2,091 functions plus 978 call trees, `--calls`) pass with this list and two other lifter changes applied to v2.0.3. - **Game A/B:** in the same batch build against stock v2.0.3, median of 3 runs each with equally warm pipeline caches, CPU per frame went down (Hyrule rotating 69.7 → 68.0 ms) and nothing regressed. That scene doesn't measure loading, so it doesn't isolate this change. Not done: a timed area-load comparison in game. 🤖 Generated with [Claude Code](https://claude.com/claude-code)