perf(host): native zstd gives its virtual core up by default, except on Intel iGPUs (lower p99 on the PCs it applies to, higher fps on the Steam Deck) #1875
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
5 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1875
Loading…
Reference in a new issue
No description provided.
Delete branch "Veix/Longshot:perf/native-zstd-give-core"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
HOS_NATIVE_ZSTDnow defaults to 1 on every PC except Intel integrated GPUs, which stay at 2. With 1, the game's zstd decode gives its virtual core up while libzstd works, so the game's other threads run meanwhile. That cuts slow frames wherever the game is short of CPU: Kokiri Forest on a Steam Deck OLED goes from 19.1 to 29.1 fps (#1726), and the Castle Town market's frames over 20 ms fall by about a third on a Ryzen desktop.Intel iGPUs keep 2 because of the load freezes on a Core Ultra 7 155H that made #1632 switch everyone to 2; that laptop hasn't been checked again since.
How: an unset
HOS_NATIVE_ZSTDasks the renderer through a new option id (HOS_OPT_NATIVE_ZSTD), which answers from the device policy (native_zstd_keep_coreingpu/vk/device_policy.h, on for an Intel integrated GPU). A value set wins as before.Read first:
zs_modeinhost/hos_zstd.c,native_zstd_keep_coreingpu/vk/device_policy.h, and where it is published (gpu/vk/vk_main.cpp) and answered (gpu/vk/vk_replay.cpp,t.option). Follows up on #1726, as zeldaootr asked there.See #1726, See #1632, See #1402, See #1477
Testing
Tested: Windows 11, Ryzen 9 9900X, RTX 4070 SUPER (NVIDIA 621.24): this PR's build in the game, six route runs (Market, Hyrule Castle, Market again, with camera turns), all ending normally; the log names the default the GPU gave (
zstd: ... (HOS_NATIVE_ZSTD=1, this GPU's default: the core given up while it decodes)).tests/test_native_zstd.py(the default by the renderer's answer, before it can answer, and a value set over it; fails on main for the new cases),tests/test_device_policy.py(the rule's edges) andtests/test_switch_table.pypass.Still to test:
HOS_NATIVE_ZSTD=2against unset, expecting no load to hang and p99 the same or better unset (@Beanstalk)HOS_NATIVE_ZSTD=2, this GPU's defaultin the log (@Beanstalk)Measurements
Verdict: the Market's frames over 20 ms halve (57 to 29) and those over 25 ms fall from 14 to 2; the 1% low goes from 40.1 to 46.3 fps, fps the same. The three runs of each side don't overlap.
Setup: Windows 11, Ryzen 9 9900X (12 cores), RTX 4070 SUPER (NVIDIA 621.24), 64 GB; 3840x2160 with DLSS Quality, 90 fps cap, V-Sync on, the frame gate and readbacks as main ships them; a recorded walk (Market, Hyrule Castle, Market again), the frame trace; 3 runs a side, interleaved, the median; the window: the second Market stay from 5 s after arriving (about 55 s), from the present intervals.
The runs were made at
65769aa3;095b160bafter it moves the rule into the device policy with no change in what a PC gets (a short run of it logsnative_zstd_keep_core=0andHOS_NATIVE_ZSTD=1, this GPU's default).Baseline: this PR's build with
HOS_NATIVE_ZSTD=2set, main's default: the only code this PR changes is which value an unset switch gets, so the same build with 2 runs as main does, and both sides share one build.65769aa3withHOS_NATIVE_ZSTD=2(main's default)65769aa3, unset (1 here)For reference, the numbers in #1726 (in-run A/B,
HOS_AB=HOS_NATIVE_ZSTD=1, both sides in the same run):=2(main's default)=1The Deck (zeldaootr): SteamOS 3.9.5, RADV, Minimum, Render 50%, uncapped. The 13900K (Jiro113): Windows 11, 2560x1440 DLAA, uncapped except the frame generation run.
The log of each run names the mode:
HOS_NATIVE_ZSTD=1, this GPU's default: the core given up while it decodesunset,HOS_NATIVE_ZSTD=2with it set. All six runs ended normally.Risks
What could break: a load that hangs where giving the core up meets whatever froze #1632's laptop on another PC; Intel integrated GPUs keep the old default for that reason. Turning it off:
HOS_NATIVE_ZSTD=2(the core kept, as main) or=0(the game's own decoder). After updating: nothing in a player's files changes.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). [tests/run_all.pyon Windows: 714 pass; the 50 that fail here fail the same onmain@89c3d21ein the same environment;test_vbytes.pytimed out at 600 s and reads none of the changed files;test_device_policy.pyskipped without the Vulkan headers and passes with them]main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.Written with help from Claude Code (Claude Opus 5.5).
Tested in game: Windows 11, Ryzen 9 9900X, RTX 4070 SUPER (NVIDIA 621.24), 4K DLSS Quality, 90 fps cap: six route runs through the Castle Town market, Hyrule Castle and the market again (3 unset, 3 with HOS_NATIVE_ZSTD=2, interleaved), all ending normally; the log names the GPU's default (HOS_NATIVE_ZSTD=1, this GPU's default); the market's frames over 20 ms 57 -> 29, no load hanging.
perf(host): the game's zstd stream decode gives its virtual core up by default, except on Intel integrated GPUs (HOS_NATIVE_ZSTD)to perf(host): native zstd gives its virtual core up by default, except on Intel iGPUs (lower p99; Steam Deck Kokiri Forest 19 to 29 fps)perf(host): native zstd gives its virtual core up by default, except on Intel iGPUs (lower p99; Steam Deck Kokiri Forest 19 to 29 fps)to perf(host): native zstd gives its virtual core up by default, except on Intel iGPUs (lower p99, higher fps on the Steam Deck)perf(host): native zstd gives its virtual core up by default, except on Intel iGPUs (lower p99, higher fps on the Steam Deck)to perf(host): native zstd gives its virtual core up by default, except on Intel iGPUs (lower p99 on the PCs it applies to, higher fps on the Steam Deck)@Beanstalk
HOS_NATIVE_ZSTD=2against unset, expecting no load to hang and p99 the same or better unsetHOS_NATIVE_ZSTD=2, this GPU's defaultin the log@Veix only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was for65769aa, and the PR has changed since (095b160), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.@Beanstalk
Beanstalk test farm · ⚠️ couldn't be judged on any
Commit
095b160· approved by llun (reviewed) · dashboardDone (started 15:07 UTC, last report 15:58 UTC). Comment
@Beanstalkto run it again.anyCouldn't be judged: the tester's PC could not finish the test. The farm tries another PC, or the same one after a fix; it says nothing about the PR on its own.
any, report 1:the main base failed (exit 1): build.py stopped (exit status 1): fatal error: too many errors emitted, stopping now [-ferror-limit=]any, report 2:Insufficient frame-time evidence for 1 required comparison(s). See rule results. a step ran out of time: timeout: not met in timeWhat was tested
What will be tested · AI plan (
gpt-6.1-terra)The plan compares the new default with main during a teleport tour from Kokiri Forest through the Market and Hyrule Castle, then measures Market frame times during a camera turn. It takes about 13 minutes per tester and checks load completion, the reported GPU default, and Market visuals. A crash, missing default-mode log, timed-out load, substantial p99 regression, or screenshot difference fails the PR.
kokiri-start, then teleport to Kokiri Forest, teleport to Market, teleport to Hyrule Castle, move the stick, take the picture market to compare with main, measure market for 30 s (after 5 s warm-up); 1 run on each side.Measurements on `any`, for the PR description
Verdict: could not be judged on any (2 reports): required frame-time comparison inconclusive (fewer than two usable starts per side).
Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 4080, 621.1.0AMD Ryzen 7 9800X3D 8-Core Processor, 32 GB RAM; 2: linux+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070, 615.71.9AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAMScene: streaming-tour, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); streaming-tour: 1 game start a side, market 30 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)89c3d21095b160Flags and settings
None: the plan varied no setting, switch or build option.
As Markdown for the PR description:
The testers' hardware
anyNVIDIA GeForce RTX 4080, 621.1.0AMD Ryzen 7 9800X3D 8-Core Processor, 32 GB RAManyNVIDIA GeForce RTX 4070, 615.71.9AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAMSteam Deck run: main against this PR, no switches set
The Deck gains from the PR's default, and no load hung. Kokiri Forest standing goes from 20.1 to 33.1 fps. The frames of 50 ms or more there fall from about 1,150 to 1 in 120 s. The Castle Town market goes from 17.5 to 20.2 fps, with a third fewer frames of 50 ms or more. Hyrule Field doesn't change while standing, and the camera pans are within run-to-run noise. With
HOS_NATIVE_ZSTD=2set on this PR's build, it runs as main does, so the switch still wins.Builds. Both builds use the same base and translator. Neither side had a switch set, and both used the same prepared shader cache, with no preparation in any run.
89c3d21ea(v3.0.0-75-g89c3d21ea).65769aa3a(v3.0.0-76-g65769aa3a), which sits directly on89c3d21ea.These runs are of
65769aa3a. The refactor in095b160b1came after them and wasn't run here.Hardware. Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH (Mesa 26.2.0-devel), GPU clock set by hand to 1600 MHz (manual before and after every run), headless 1280x800.
Settings. Developers' settings: Minimum (
quality_preset=1),handheld_profile=0, Render 50% (resolution=4), 1280x800, frame pacing on, uncapped (frame_rate_cap=6), V-Sync off, save at the Kokiri start.HOS_NATIVE_ZSTD=2set.Measures. Each table gives fps, p99 frame time, the 1% low (1000 / the mean of the slowest 1% of frame times) and the number of frames of 50 ms or more in the window. Each run is shown, with the median in bold.
Kokiri Forest, standing 120 s
HOS_NATIVE_ZSTD=2Hyrule Field, standing 60 s after the teleport
=2Hyrule Field, 4 camera pans (about 44 s)
=2The runs of the two sides overlap here, so the pans show no difference either way.
Hyrule Field, standing after the pans (about 87 s)
=2In this window, B's fps is a little higher, and the runs of the two sides don't overlap.
Castle Town market, standing 120 s
At the market, fps and the frames of 50 ms or more don't overlap between the sides. The p99 and the 1% low do overlap, and the market's longest stalls (over 100 ms) are on both sides.
The mode each side logged
Each run logged its mode once, at line 115 of the log (logs of 5,400 to 8,800 lines). That is before the first presents line (lines 471 to 481), so the mode was settled before the first frames.
[hos ParallelDecompThread] zstd: the game's stream decode on the host's libzstd 1.5.7 (HOS_NATIVE_ZSTD=2)[hos ParallelDecompThread] zstd: the game's stream decode on the host's libzstd 1.5.7 (HOS_NATIVE_ZSTD=1, this GPU's default: the core given up while it decodes)HOS_NATIVE_ZSTD=2set:[hos ParallelDecompThread] zstd: the game's stream decode on the host's libzstd 1.5.7 (HOS_NATIVE_ZSTD=2), without "this GPU's default"The override check. B with
=2set matches A in every window: Kokiri 20.15 fps against A's 20.1, and 1139 frames of 50 ms or more against A's 1146. A value set still wins over the GPU's default.Loads and errors. All 9 runs exited with code 0. No run logged
VK_ERROR, device lost, a validation message, an[error]line or a crash. No load hung:=2.Two questions on the diff
gpu/vk/switches.defchanges the switch's kind fromattempttohw:hwlooks right for a default that now follows the GPU. The old row's "off" was out of date on main too, where the default is 2. Could the description say that the switch's kind changes, so a reader of the switch table isn't surprised?095b160b1: at65769aa3a, the comment overnative_zstd_modeingpu/vk/device_policy.hquotedthe Castle Town market's frames over 20 ms 42.5 -> 26 (Ryzen 9 9900X, RTX 4070 SUPER), while the Measurements give 57 -> 29. That comment is gone at the current head, so there is nothing to change. I mention it only in case those numbers are reused elsewhere.Limits
65769aa3a, not the current head095b160b1, whose description says it gives every PC the same mode.zeldaootr
Linux results from another PC, for the "linux" item in Still to test: Linux (CachyOS, kernel 7.3), RTX 3090 (NVIDIA 620.x), Ryzen 9 7900X (12 cores), 2560x1440, uncapped, Threaded rendering Auto. This PR @
095b160b, one build: unset (logsHOS_NATIVE_ZSTD=1, this GPU's default: the core given up while it decodes) againstHOS_NATIVE_ZSTD=2. Kokiri Forest then Castle Town Market, each 20 s standing, 15 s walking, 15 s turning; four rounds, the order alternated each round (the PC ran slower as the session went on, so compare within a round). No hang or crash; all loads ended normally.=2(main) runs=1) runs=2/=1The Market standing was lower unset in all four rounds (by 1-6%, about -3.5%); walking p99 a little worse, turning p99 a little better; Kokiri even. So on this 12-core Linux PC giving the core up doesn't help and costs a little where the game streams.
A guess at why, not measured: giving the virtual core up pays where the game's virtual cores are contended (the Deck's 4 cores, a busy PC), and costs a re-acquire on each decode where they aren't (12 cores here). If that holds, the default might follow the physical core count (
usable_cores, as Threaded rendering's Auto does) rather than the GPU: 1 on few cores, 2 on many. The vcores log lines of both sides would show whether the decode thread waits more for its core unset.On Windows with 12 cores, standing in the Market costs nothing with
=1, even CPU-bound.Windows 11, Ryzen 9 9900X, RTX 4070 SUPER; main
2210931c+ this PR, no switches besidesHOS_NATIVE_ZSTD; Market standing 60 s (from +5 s), one run each:=1)=2No frame over 20 ms in any of them. The gain here is all in moving (the route in Measurements: frames over 20 ms 57 to 29). So the standing cost on the 7900X may be Linux-specific (the host's pthread emulation of the waits, or the scheduler) rather than the core count. Happy to re-test the conditional version on this PC once it's on the branch.
A caveat on my Steam Deck numbers above: they came from a game folder with three damaged romfs files, and that is where most of the Kokiri and Market cost comes from.
What I found since, on the same Deck:
HyruleField_Child. One model in that preload,Land_FairyFountain_Base_A_01, never loads: its textureTex/CmnTex_Map_Soil_FairyFountain_Floor_D_Nrm.bntx.zsstarts with 12,288 zero bytes in my dump, so it is not a zstd frame. The game decodes the 10.8 MiB model, fails on the texture, lets the handle go and asks again, about 15 times a second, for as long as Link stands there. That is the loader at ~80 % of a core and the decode threads at ~55 % that the Deck's logs showed, and it is whatHOS_NATIVE_ZSTD=2was holding virtual cores for.nx2/nup.pyalready knows these files ("in the 23.3 GB dumps three of them begin with zeros") andmake_game.stand_in_damaged_filesputs stand-ins in, but only in a romfs the builder unpacked itself (the.longshot_romfsstamp). My romfs was unpacked by Hookshot, has no stamp, and kept all three: that one,Tex/Rock_WaterTemple_Gate_M_01_Nrm.bntx.zsandTex/RBake_Scene_GanonStage_G_2_1_d0_t0.bntx.zs. Nothing in the log says so.With the builder's own stand-ins served through
HOS_FS_OVERLAY(the same build, main2210931cfplus a trace switch,HOS_NATIVE_ZSTD=1on both sides, Steam Deck OLED, Minimum, Render 50%, uncapped, headless, 60 s standing, two starts a side, interleaved):With the stand-ins the preload finishes in about 10 s and the loader idles, as it does in Hyrule Field.
What this means for this PR, as far as my data goes:
=2, Kokiri gave 19.2 fps and 634 frames of 50 ms or more; with=1, about 30. So=1still protects anyone whose romfs has this problem, and I have no reason to think it costs anything.=1against=2on a repaired romfs. With the loader idle while standing, I would expect the difference there to be small and to show mainly at area changes. I will run that and post it here.romfs/Tex/CmnTex_Map_Soil_FairyFountain_Floor_D_Nrm.bntx.zsshould be28 b5 2f fd, not zeros.I will open an issue for the missing warning (a romfs without the builder's stamp keeps the damaged files silently).
zeldaootr
HOS_NATIVE_ZSTD=1against=2with the damaged romfs files replaced (Steam Deck OLED)As promised: the earlier Kokiri result (=2 → =1: 20 → 33 fps) was measured on a romfs with three damaged textures. One of them keeps the game's background preloader retrying a 10.8 MB model forever in Kokiri Forest and the Castle Town Market, which costs about 1.3 cores. That loop decodes zstd nonstop, so it is where the zstd mode matters most. This run measures both modes again with the damaged files replaced.
Short answer: with the files replaced,
=1and=2measure the same within the spread of 4 starts a side in every window. On the damaged data, measured the same day by the same tool,=1is far ahead (Kokiri +57%, Market +41%). So=1never measured worse here, and it matters a lot on a romfs like ours. Nothing here argues against the PR's default of 1.What ran
89c3d21eaplus this PR at65769aa3a(logslongshot v3.0.0-76-g65769aa3a). Both arms run the same build. A leavesHOS_NATIVE_ZSTDunset, which logsHOS_NATIVE_ZSTD=1, this GPU's default. B setsHOS_NATIVE_ZSTD=2, which logsHOS_NATIVE_ZSTD=2.HOS_THREAD_TIMES=5. Warm caches. Starts from the Kokiri-start save.nup.stand_in_for) throughHOS_FS_OVERLAY. The switches line of every start lists it. In every window and every start of both arms, the loader thread (OptimizeChangeBank) used 0.1% of a core and the decode threads 0.0%. On the damaged data they used 78–85% and 47–56% in Kokiri and the Market.tour3: each start covers Kokiri (45 s stand), then Hyrule Field by teleport (20 s stand, 4 camera pan cycles, 30 s stand) and the Market by teleport (60 s from 20 s after arrival). Every other round visits the Market before the Field. Starts run A B B A.Repaired data: 4 starts a side
=1(unset) each start=2each start=1→=2(median)=1→=2(median)All five windows are within the spread. In every window the two arms' starts overlap, and no difference is larger than this job's own spread. The pans lean towards
=1: each=2start is below the=1start with the same route order, by -6.0% on the medians. The Market leans towards=2(+4.1%). Neither is resolved with 4 starts a side.Load times (teleport to the place reached and settled; polled every 1 s after a fixed 3 s, so they resolve about 1 s):
=1=2There is no measurable difference in load time either.
Damaged data, same day and tool: 2 starts a side, no overlay
=1each start (median)=2each start (median)With the retry loop,
=2costs a third of the fps in Kokiri and the Market, with no overlap even at 2 starts a side. With the files repaired, the difference goes away.What N starts can resolve
The fps differences these numbers can resolve are estimated from 769 earlier starts on this Deck. With 4 starts a side: Kokiri about 6–10%, Field stand 1 about 1–5%, pans about 3–8%, Field stand 2 about 11–14%, Market about 4–12%. With 2 starts a side: about 8–14%, 1–7%, 5–11%, 15–20% and 5–16%.
The job's own spread was 1.9% in Kokiri, 10.1% in Field stand 1, 7.9% in the pans, 11.4% in Field stand 2 and 9.1% in the Market. The Field stands' spread is mostly route order: they are slower when the Field follows the Market. The spread is wide enough to hide the -6% in the pans and the +4% in the Market. By the same estimate, resolving 5% would take about 2–10 starts a side in the pans and 3–22 in the Market. With the spread seen here, expect the top of those ranges.
Limits
— zeldaootr