feat(host): HOS_NATIVE_ZSTD in the in-run A/B #1726

Open
zeldaootr wants to merge 2 commits from zeldaootr/Longshot:ab-native-zstd into main
Contributor

Summary

Whether a native zstd decode keeps its virtual core (HOS_NATIVE_ZSTD=2) or gives it up (=1) could only be compared across separate runs, which differ by more than most changes are worth: HOS_NATIVE_ZSTD was read once, so HOS_AB could not alternate it. This PR lets the in-run A/B flip that rule, so both sides share one run and one scene state: HOS_AB=HOS_NATIVE_ZSTD=<1 or 2>,<seconds>,<warm-up> alternates the mode in force with that value. It is for developers and testers measuring the zstd core rule on their own hardware. The default, every run without HOS_AB, the decoder, and HOS_NATIVE_ZSTD=0 (the lifted decoder, which cannot be flipped) do not change.

Since #1875 the default comes from the renderer's device policy while the switch is unset: 1 (the core given up) on every GPU except Intel integrated GPUs, which keep 2; a value set still wins. The A/B flips whichever mode is in force, set or the GPU's default, as hos_ab_value does for the other wired switches (their normal value is the one in force, default included): on a GPU whose default is 1, HOS_AB=HOS_NATIVE_ZSTD=2 measures keeping the core against it.

Why: #1632 made the native zstd decode keep its virtual core, which fixed load freezes on an Arc 155H. On a Steam Deck OLED that default costs about a third of the frame rate where the game decodes in the background (Kokiri Forest in our runs), and another Deck sees no difference because its scene is not decoding. Separate runs can't settle that kind of question cheaply; an in-run A/B can. Still open: why Kokiri decodes in the background on one Deck and not on another (#1477).

Read first:

  • zs_core() in host/hos_zstd.c returns hos_ab_value("HOS_NATIVE_ZSTD", mode); hos_zstd_stream uses it to decide whether to wrap the chunk in hos_vcore_away / hos_vcore_back. An A/B value other than 1 or 2 is taken as 2 and logged once. zs_mode(), the on/off choice and the mode in force (the value set, else the GPU's default through HOS_OPT_NATIVE_ZSTD), is unchanged and still read once; until the renderer can answer, a call keeps the core (2) and the A/B flips that.
  • Why 1 and 2 are safe to flip: the rule only decides whether the call gives the core up and takes it back, within itself; the stream's state is the same either way. Why 0 is not: a stream begun on libzstd has no state in the game's workspace for the lifted decoder to go on from.
  • gpu/vk/switches.def: the HOS_NATIVE_ZSTD row (main's hw row, its default by GPU) gains a sentence: HOS_AB may flip it between 1 and 2, not to 0 (read once: dev.py ab --separate). Main's developing guide points to the rows ("HOS_AB may flip it") and to tests/test_hos_ab.py rather than listing the A/B's switches in prose, so that row is this PR's doc text now.
  • tests: tests/test_hos_ab.py lists the switch among the wired ones; tests/test_native_zstd.py decodes streams with the value flipping at every other call and checks each call's core hand-over and the output, with the mode set (2 against 1, 1 against 2) and unset (the renderer's 1 against 2, its 2 against 1, and 2 against 1 before it can answer), plus a broken copy that never reads the A/B.

See #1477

Testing

Tested: in the game on a Steam Deck OLED (SteamOS 3.9.5, RADV) with the A/B on, at 1b15e8a4d (this PR's patch on main before #1875, when main's default was 2; later rebases moved only doc and comment lines, and the one onto 3df5c13ed keeps zs_core as it was, now over the GPU's default): five runs of 340 to 460 s, two standing in Kokiri Forest, two in Hyrule Field (standing, then camera pans), one in the Castle Town market (standing, then a camera-pan cycle). All five ended normally (exit 0, no error lines) and the sides alternated every 5 s; the Kokiri and market logs' closing ab: lines have no "never read" note, and the Field's say the switch was never read during the A/B, as expected where nothing is stream-decoded.

On Windows 11 (Intel Core i9-13900K, NVIDIA GeForce RTX 4090, driver 621.24, 64 GB), by Jiro113 (#1726, their comment of 2026-11-19 13:58): built with llvm-mingw at c03bcb70 plus the two commits of #1729 (main did not build on Windows without them then; this PR's host/ and nx2/ are unchanged by them), with tests/test_native_zstd.py (zstandard 0.25.0), tests/test_hos_ab.py, tests/test_switch_table.py and tests/test_doc_anchors.py passing there, run one by one. In the game, HOS_AB=HOS_NATIVE_ZSTD=1,3,60 and RX_VK_FRAME_TRACE=1, three minutes walking in Castle Town's south market, closed through the control bridge: a closing ab: line with frames on both sides and no "never read" note (side 0 54.7 fps, side 1 59.3 fps, -7.71 % at a 60 fps ceiling: DLSS Frame Generation 2x with V-Sync Auto on a 120 Hz screen), plus the two uncapped runs in Measurements.

Game-free, on Linux x86-64, at this head (7c5cf948a on main 3df5c13ed): tests/test_hos_ab.py and tests/test_native_zstd.py fail at the test-only commit cbd3c4389 (every new A/B case but =2 against 0, which keeps the core either way) and pass here; tests/test_native_zstd.py run with the zstandard module (0.25.0): every part PASS. The 27 checks that read host/hos_zstd.c or gpu/vk/switches.def, plus tests/test_device_policy.py (28 in all), here and on main 3df5c13ed in a clean worktree, same environment: 21 pass, 7 fail on both with the same messages (test_crash_dialog, test_dev_ab, test_frame_overlap, test_linux_pressure, test_present_queue_default, test_target_reuse_memory, test_vk_occlusion_queries: this PC's environment); test_native_zstd, test_hos_ab, test_switch_table and test_device_policy pass on both. host/hos_zstd.c compiled (-c, the build's host flags, -Wall) with gcc and clang for Linux and llvm-mingw 20261104 for Windows, at this head and on main 3df5c13ed: no warnings on either.

The whole python3 -I -B tests/run_all.py (not --strict) was run at the previous head (88ff71239 on main 068fb8ebe), not again at this one: 74 failed both there and on main 068fb8ebe in the same environment (missing tools and modules, timing); 12 more failed in a run shared with two other full suites and each passed when run again alone (nx2/test_coop_spirv.py, nx2/test_default_point_size.py, nx2/test_draw_parameters.py, nx2/test_mesh_first_types.py, nx2/test_spirv_struct_identity.py, test_appimage_package_retry, test_fpx_sticky, test_mesh_fallback_layout, test_mesh_unreachable_outputs, test_pipeline_cache_file, test_pipeline_cache_save, test_pipeline_state_sources).

The 74 checks that failed both at the previous head (88ff71239) and on main 068fb8ebe

nx2/test_nup_codec.py, nx2/test_nup_read_name.py, nx2/test_nup_stand_in.py, nx2/test_nup_verify.py, test_appimage_player_build, test_bug_report_defaults, test_capture_read_once, test_capture_readback_visibility, test_crash_dialog, test_crt_options, test_damage_debug_menu, test_dav1d_av1, test_dev_ab, test_dev_packs, test_discover, test_display_startup_failure, test_dlss_feature_size, test_draw_stream_visibility, test_failed_compile_keeps_objects, test_frame_overlap, test_fsr3fi_hud_layer, test_fsr_image_retirement, test_fsr_uniform_ownership, test_guest_imported, test_high_fps_fixes, test_host_guest_writes, test_hud_layout_row, test_insn_fixed_convert, test_libc_hooks, test_lift_cache, test_lift_lookups, test_linux_preflight, test_linux_pressure, test_linux_tools, test_linux_update, test_log_open_failure, test_make_game_cache, test_make_game_ngx, test_mise_build, test_missing_nup, test_mods_list, test_mods_size_table, test_native_zstd, test_ngx_sdk, test_nonuniform_image_arrays, test_parallel_init_order, test_pending_guest_import, test_perf_report, test_pipeline_play_cache, test_prepare_cache_save, test_prepare_stall, test_present_queue_default, test_present_source_crop, test_present_thread, test_profiling_mode, test_prompt_guide_lines, test_prompt_hand_guides, test_prompt_picture_tint, test_prompt_speed, test_psychov35_fsr3_fg, test_readback_timeout_source, test_recorder_sync, test_replay_write_order, test_slow_frames, test_song_note_cue, test_staging_burst, test_texel_buffer_ownership, test_thinlto_tls, test_upscaler_rows, test_vbytes, test_vcore_escape, test_vk_occlusion_queries, test_walk, test_walk_key

Still to test:

  • Windows on AMD or Intel graphics, and Windows outside the market: an A/B run as above, expecting the closing ab: line with frames on both sides (by hand)

Measurements

Verdict: the A/B works and resolves the rule's cost in one run, scene by scene. On a Steam Deck OLED, side 1 (=1, the core given up) against side 0 (2, main's default when these runs were made): standing in Kokiri Forest 34.4 % less frame time, 19.1 against 29.1 fps (two runs); in the Castle Town market 15.7 % less, 14.7 against 17.4 fps (one run); in Hyrule Field no difference, 33.9 against 34.0 fps (two runs), because nothing is stream-decoded there and the switch was never read. On Windows with an RTX 4090 (Jiro113) the gap is far smaller and two runs don't settle it. This PR changes no default, so it has no before/after of its own: the table's columns are the A/B's two sides in the same runs.

These runs were taken with this PR on main before #1875, whose default was 2. On current main (3df5c13ed) the default on these PCs (a Steam Deck, an RTX 4090) is 1, so the same comparison is HOS_AB=HOS_NATIVE_ZSTD=2 with the sides swapped; the A/B code is unchanged, and #1875 cites these numbers beside its own.

Baseline: both columns are this PR's build (1b15e8a4d, v3.0.0-5-g1b15e8a4d) within the same runs: the before side is the rule in force, then main's default (HOS_NATIVE_ZSTD 2), the after side HOS_NATIVE_ZSTD=1. That is the comparison this PR exists for; main cannot alternate the rule. 1b15e8a4d is this PR's patch before the rebases (onto 868f6be87 and now 068fb8ebe), which moved only doc and comment lines: the code is the same.
Setup: Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH (Mesa 26.2.0-devel 5c65d4fc31), AMD Custom APU 0932, 16 GB, GPU clock set by hand to 1600 MHz; Kokiri Forest standing, daytime; Hyrule Field after a teleport from Kokiri, 30 s standing then camera pans; the Castle Town market (child) after a teleport, standing, then one camera-pan cycle; 1280x800, Minimum, Render 50%, the game's own upscaler, sharpness 30, frame generation off, frame pacing on, no cap; headless, fresh options, no longshot_env.txt. RX_VK_FRAME_TRACE=1 HOS_AB=HOS_NATIVE_ZSTD=1,5,150 (5 s periods after 150 s); five runs (Kokiri 2, Field 2, market 1), each about 300 s of A/B; scripts/ab_summary.py on the frame traces (both runs of a scene together, transition frames left out, bootstrap interval with seed 1) and each log's closing ab: line. To reproduce: a save standing in Kokiri Forest (or teleport to the market or Hyrule Field), the settings above, the two switches in longshot_env.txt, then python -I scripts/ab_summary.py <game folder>/logs/longshot_frames.csv.

Before, HOS_NATIVE_ZSTD 2 (side 0, then the default): this PR @ 1b15e8a4d After, HOS_NATIVE_ZSTD=1 (side 1): this PR @ 1b15e8a4d Δ
Kokiri Forest, both runs (120 periods): mean ms 52.37 34.37 -18.00 ms (-34.37 %), 95 % interval -18.78 to -17.19
Kokiri Forest: fps 19.1 29.1 +52.4 %
Castle Town market, one run (57 periods): mean ms 67.98 57.34 -10.65 ms (-15.66 %), 95 % interval -13.17 to -8.05
Castle Town market: fps 14.7 17.4 +18.6 %
Hyrule Field, both runs (74 periods): mean ms 29.48 29.43 -0.05 ms (-0.17 %), 95 % interval -0.69 to +0.61 (no difference)
Hyrule Field: fps 33.9 34.0 +0.2 %

Each run's sides:

Run Side 0 frames Side 0 mean / median / p99 (ms) Side 1 frames Side 1 mean / median / p99 (ms)
Kokiri 1 2,767 52.72 / 51.87 / 72.19 4,115 34.28 / 33.29 / 52.07
Kokiri 2 2,711 52.01 / 51.26 / 70.98 3,955 34.46 / 33.61 / 49.41
Market 1 2,008 67.98 / 62.74 / 167.72 2,296 57.34 / 52.36 / 157.51
Field 1 2,983 29.92 / 28.86 / 53.46 2,883 29.34 / 28.00 / 51.71
Field 2 3,075 29.06 / 28.10 / 48.55 2,864 29.53 / 28.53 / 50.64

The logs' closing lines (the host's own count, before ab_summary.py leaves out the periods' edges):

Kokiri 1: ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 2767 frames, 52.705 ms a frame (19.0 fps); side 1 (the A/B's value) 4121 frames, 34.278 ms a frame (29.2 fps); side 1 -34.96 % frame time; 438 frames in transitions left out
Kokiri 2: ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 2712 frames, 52.014 ms a frame (19.2 fps); side 1 (the A/B's value) 3962 frames, 34.455 ms a frame (29.0 fps); side 1 -33.76 % frame time; 423 frames in transitions left out
Market 1: ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 2014 frames, 68.101 ms a frame (14.7 fps); side 1 (the A/B's value) 2378 frames, 57.324 ms a frame (17.4 fps); side 1 -15.83 % frame time; 291 frames in transitions left out
Field 1: ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 3059 frames, 29.880 ms a frame (33.5 fps); side 1 (the A/B's value) 3032 frames, 29.444 ms a frame (34.0 fps); side 1 -1.46 % frame time; 397 frames in transitions left out (...); HOS_NATIVE_ZSTD was never read during the A/B: no switch of that name takes part, both sides ran the same
Field 2: ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 3075 frames, 29.057 ms a frame (34.4 fps); side 1 (the A/B's value) 3018 frames, 29.515 ms a frame (33.9 fps); side 1 +1.57 % frame time; 388 frames in transitions left out (...); HOS_NATIVE_ZSTD was never read during the A/B: no switch of that name takes part, both sides ran the same

The Field's "never read" note is the A/B working as it should: no stream decode happened there during the A/B, so both sides ran the same code and the difference is noise. The gap exists only while the game decodes in the background: another Steam Deck OLED saw none standing in Kokiri, with its decode threads idle (#1477). In the market run MemAvailable fell to 576 MiB at its lowest (one sample under 768 MiB), in a run both sides shared; in Kokiri and the Field it stayed above 1.6 GiB. All five runs exited 0 with no error lines. The Kokiri and the Field have two runs each; the market one run of 57 interleaved periods.

On Windows (Jiro113's runs, their numbers): Windows 11, Intel Core i9-13900K, NVIDIA GeForce RTX 4090 (driver 621.24), 64 GB; this PR at c03bcb70 with the two commits of #1729. Castle Town's south market (child), walking a scripted route through the control bridge, 2560x1440 windowed, HDR10, DLAA, frame generation off, V-Sync off, Reflex on, frame pacing on, no cap (HOS_FPS_MAX=1000); RX_VK_FRAME_TRACE=1 HOS_AB=HOS_NATIVE_ZSTD=1,5,150, two runs of about 315 s of A/B (126 periods together). Over both runs side 1 (=1) has 4.39 % less mean frame time, 65.7 against 62.8 fps, with a 95 % interval of -9.09 to +0.23 % that holds 0: the gap is far smaller than on the Deck and two runs don't settle it (run 2 alone -5.59 %, interval -10.81 to -0.91 %; run 1 -3.18 %, interval -11.70 to +4.50 %). The tail is shorter with 1 (p99 37.15 against 28.51 ms in run 1, 34.38 against 24.04 in run 2); the median is a little longer with 1 (14.31 against 14.66 ms over both).

Jiro113, Windows 11, RTX 4090, market walk, uncapped Side 0: HOS_NATIVE_ZSTD 2 (then the default) Side 1: HOS_NATIVE_ZSTD 1
Run 1: frames 9,511 9,524
Run 1: mean / median / p99 (ms) 15.811 / 14.020 / 37.150 15.308 / 14.560 / 28.509
Run 2: frames 9,373 9,621
Run 2: mean / median / p99 (ms) 16.041 / 14.560 / 34.376 15.145 / 14.730 / 24.040
Both runs, 126 periods: mean (ms) 15.925 15.226
Both runs: fps 62.8 65.7
Side 1 less side 0 -0.699 ms (-4.39 %), 95 % interval -9.09 to +0.23 %
ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 9507 frames, 15.811 ms a frame (63.2 fps); side 1 (the A/B's value) 9740 frames, 15.322 ms a frame (65.3 fps); side 1 -3.09 % frame time; 1245 frames in transitions left out (the host's presents; scripts/ab_summary.py on the frame trace for medians and a confidence interval)
ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 9379 frames, 16.042 ms a frame (62.3 fps); side 1 (the A/B's value) 9792 frames, 15.163 ms a frame (66.0 fps); side 1 -5.48 % frame time; 1248 frames in transitions left out (the host's presents; scripts/ab_summary.py on the frame trace for medians and a confidence interval)

The switch: no new switch; HOS_NATIVE_ZSTD (2 keeps the virtual core, 1 gives it up, 0 the lifted decoder) ships at main's default, unchanged (unset: the GPU's, 1 except on Intel integrated GPUs, which keep 2; a value set wins), and is still read once without HOS_AB; with it, the A/B may alternate the mode in force with 1 or 2. Measured on a Steam Deck OLED (Kokiri, the market, Hyrule Field) and on Windows 11 with an RTX 4090 (the market, by Jiro113). Off (no HOS_AB), the added cost is one load a decode call; it was not timed apart.

Risks

A flip between two decode calls of one stream could break a decode; the test runs exactly that at every other call, and the rule touches nothing the stream keeps. Without HOS_AB nothing changes, so it is off unless a developer turns it on; a revert is enough to undo it. A player notices nothing after updating: saves, longshot_options.ini and the caches are untouched.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says. [a Steam Deck OLED at 1b15e8a4d, the same code]
  • mise run test passes (any failure that also fails on main is named). [at this head: the checks that read the changed files, the same as on main 3df5c13ed; the whole suite at the previous head, Testing]
  • Measured against main, or Measurements says why there is nothing to measure. [the A/B's two sides in the same runs; main cannot alternate the rule: Baseline]
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A switch added or changed: it ships at the default the measurements chose, with its row in gpu/vk/switches.def. [HOS_NATIVE_ZSTD's default, the GPU's from the device policy, unchanged; its row says what the A/B may flip]
  • A check under tests/ that reads changed source is updated, not weakened.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. [built and run on Windows by Jiro113 at c03bcb70; host/hos_zstd.c compiled for both at this head]
  • Docs updated. [the HOS_NATIVE_ZSTD row in gpu/vk/switches.def]
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested. [Windows on AMD or Intel graphics not run]
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). [the Steam Deck and an NVIDIA RTX 4090 on Windows; AMD RX 6000-9000 and Linux NVIDIA not run]
  • Approved in review, every comment answered. [no review yet]
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold. [the head is not pushed yet; the numbers are from 1b15e8a4d, the same A/B code, on main before #1875 (default 2): Measurements says how they map onto the new default]
## Summary Whether a native zstd decode keeps its virtual core (`HOS_NATIVE_ZSTD=2`) or gives it up (`=1`) could only be compared across separate runs, which differ by more than most changes are worth: `HOS_NATIVE_ZSTD` was read once, so `HOS_AB` could not alternate it. This PR lets the in-run A/B flip that rule, so both sides share one run and one scene state: `HOS_AB=HOS_NATIVE_ZSTD=<1 or 2>,<seconds>,<warm-up>` alternates the mode in force with that value. It is for developers and testers measuring the zstd core rule on their own hardware. The default, every run without `HOS_AB`, the decoder, and `HOS_NATIVE_ZSTD=0` (the lifted decoder, which cannot be flipped) do not change. Since https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1875 the default comes from the renderer's device policy while the switch is unset: 1 (the core given up) on every GPU except Intel integrated GPUs, which keep 2; a value set still wins. The A/B flips whichever mode is in force, set or the GPU's default, as `hos_ab_value` does for the other wired switches (their normal value is the one in force, default included): on a GPU whose default is 1, `HOS_AB=HOS_NATIVE_ZSTD=2` measures keeping the core against it. Why: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1632 made the native zstd decode keep its virtual core, which fixed load freezes on an Arc 155H. On a Steam Deck OLED that default costs about a third of the frame rate where the game decodes in the background (Kokiri Forest in our runs), and another Deck sees no difference because its scene is not decoding. Separate runs can't settle that kind of question cheaply; an in-run A/B can. Still open: why Kokiri decodes in the background on one Deck and not on another (https://git.magicbean.dev/MagicBeanDev/Longshot/issues/1477). Read first: - `zs_core()` in `host/hos_zstd.c` returns `hos_ab_value("HOS_NATIVE_ZSTD", mode)`; `hos_zstd_stream` uses it to decide whether to wrap the chunk in `hos_vcore_away` / `hos_vcore_back`. An A/B value other than 1 or 2 is taken as 2 and logged once. `zs_mode()`, the on/off choice and the mode in force (the value set, else the GPU's default through `HOS_OPT_NATIVE_ZSTD`), is unchanged and still read once; until the renderer can answer, a call keeps the core (2) and the A/B flips that. - Why 1 and 2 are safe to flip: the rule only decides whether the call gives the core up and takes it back, within itself; the stream's state is the same either way. Why 0 is not: a stream begun on libzstd has no state in the game's workspace for the lifted decoder to go on from. - `gpu/vk/switches.def`: the `HOS_NATIVE_ZSTD` row (main's `hw` row, its default by GPU) gains a sentence: `HOS_AB` may flip it between 1 and 2, not to 0 (read once: `dev.py ab --separate`). Main's developing guide points to the rows ("HOS_AB may flip it") and to `tests/test_hos_ab.py` rather than listing the A/B's switches in prose, so that row is this PR's doc text now. - tests: `tests/test_hos_ab.py` lists the switch among the wired ones; `tests/test_native_zstd.py` decodes streams with the value flipping at every other call and checks each call's core hand-over and the output, with the mode set (2 against 1, 1 against 2) and unset (the renderer's 1 against 2, its 2 against 1, and 2 against 1 before it can answer), plus a broken copy that never reads the A/B. See https://git.magicbean.dev/MagicBeanDev/Longshot/issues/1477 ## Testing **Tested:** in the game on a Steam Deck OLED (SteamOS 3.9.5, RADV) with the A/B on, at 1b15e8a4d (this PR's patch on main before https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1875, when main's default was 2; later rebases moved only doc and comment lines, and the one onto 3df5c13ed keeps `zs_core` as it was, now over the GPU's default): five runs of 340 to 460 s, two standing in Kokiri Forest, two in Hyrule Field (standing, then camera pans), one in the Castle Town market (standing, then a camera-pan cycle). All five ended normally (exit 0, no error lines) and the sides alternated every 5 s; the Kokiri and market logs' closing `ab:` lines have no "never read" note, and the Field's say the switch was never read during the A/B, as expected where nothing is stream-decoded. On Windows 11 (Intel Core i9-13900K, NVIDIA GeForce RTX 4090, driver 621.24, 64 GB), by Jiro113 (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1726, their comment of 2026-11-19 13:58): built with llvm-mingw at c03bcb70 plus the two commits of https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1729 (`main` did not build on Windows without them then; this PR's `host/` and `nx2/` are unchanged by them), with `tests/test_native_zstd.py` (`zstandard` 0.25.0), `tests/test_hos_ab.py`, `tests/test_switch_table.py` and `tests/test_doc_anchors.py` passing there, run one by one. In the game, `HOS_AB=HOS_NATIVE_ZSTD=1,3,60` and `RX_VK_FRAME_TRACE=1`, three minutes walking in Castle Town's south market, closed through the control bridge: a closing `ab:` line with frames on both sides and no "never read" note (side 0 54.7 fps, side 1 59.3 fps, -7.71 % at a 60 fps ceiling: DLSS Frame Generation 2x with V-Sync Auto on a 120 Hz screen), plus the two uncapped runs in Measurements. Game-free, on Linux x86-64, at this head (7c5cf948a on main 3df5c13ed): `tests/test_hos_ab.py` and `tests/test_native_zstd.py` fail at the test-only commit cbd3c4389 (every new A/B case but `=2` against 0, which keeps the core either way) and pass here; `tests/test_native_zstd.py` run with the `zstandard` module (0.25.0): every part PASS. The 27 checks that read `host/hos_zstd.c` or `gpu/vk/switches.def`, plus `tests/test_device_policy.py` (28 in all), here and on main 3df5c13ed in a clean worktree, same environment: 21 pass, 7 fail on both with the same messages (`test_crash_dialog`, `test_dev_ab`, `test_frame_overlap`, `test_linux_pressure`, `test_present_queue_default`, `test_target_reuse_memory`, `test_vk_occlusion_queries`: this PC's environment); `test_native_zstd`, `test_hos_ab`, `test_switch_table` and `test_device_policy` pass on both. `host/hos_zstd.c` compiled (`-c`, the build's host flags, `-Wall`) with gcc and clang for Linux and llvm-mingw 20261104 for Windows, at this head and on main 3df5c13ed: no warnings on either. The whole `python3 -I -B tests/run_all.py` (not `--strict`) was run at the previous head (88ff71239 on main 068fb8ebe), not again at this one: 74 failed both there and on main 068fb8ebe in the same environment (missing tools and modules, timing); 12 more failed in a run shared with two other full suites and each passed when run again alone (`nx2/test_coop_spirv.py`, `nx2/test_default_point_size.py`, `nx2/test_draw_parameters.py`, `nx2/test_mesh_first_types.py`, `nx2/test_spirv_struct_identity.py`, `test_appimage_package_retry`, `test_fpx_sticky`, `test_mesh_fallback_layout`, `test_mesh_unreachable_outputs`, `test_pipeline_cache_file`, `test_pipeline_cache_save`, `test_pipeline_state_sources`). <details><summary>The 74 checks that failed both at the previous head (88ff71239) and on main 068fb8ebe</summary> `nx2/test_nup_codec.py`, `nx2/test_nup_read_name.py`, `nx2/test_nup_stand_in.py`, `nx2/test_nup_verify.py`, `test_appimage_player_build`, `test_bug_report_defaults`, `test_capture_read_once`, `test_capture_readback_visibility`, `test_crash_dialog`, `test_crt_options`, `test_damage_debug_menu`, `test_dav1d_av1`, `test_dev_ab`, `test_dev_packs`, `test_discover`, `test_display_startup_failure`, `test_dlss_feature_size`, `test_draw_stream_visibility`, `test_failed_compile_keeps_objects`, `test_frame_overlap`, `test_fsr3fi_hud_layer`, `test_fsr_image_retirement`, `test_fsr_uniform_ownership`, `test_guest_imported`, `test_high_fps_fixes`, `test_host_guest_writes`, `test_hud_layout_row`, `test_insn_fixed_convert`, `test_libc_hooks`, `test_lift_cache`, `test_lift_lookups`, `test_linux_preflight`, `test_linux_pressure`, `test_linux_tools`, `test_linux_update`, `test_log_open_failure`, `test_make_game_cache`, `test_make_game_ngx`, `test_mise_build`, `test_missing_nup`, `test_mods_list`, `test_mods_size_table`, `test_native_zstd`, `test_ngx_sdk`, `test_nonuniform_image_arrays`, `test_parallel_init_order`, `test_pending_guest_import`, `test_perf_report`, `test_pipeline_play_cache`, `test_prepare_cache_save`, `test_prepare_stall`, `test_present_queue_default`, `test_present_source_crop`, `test_present_thread`, `test_profiling_mode`, `test_prompt_guide_lines`, `test_prompt_hand_guides`, `test_prompt_picture_tint`, `test_prompt_speed`, `test_psychov35_fsr3_fg`, `test_readback_timeout_source`, `test_recorder_sync`, `test_replay_write_order`, `test_slow_frames`, `test_song_note_cue`, `test_staging_burst`, `test_texel_buffer_ownership`, `test_thinlto_tls`, `test_upscaler_rows`, `test_vbytes`, `test_vcore_escape`, `test_vk_occlusion_queries`, `test_walk`, `test_walk_key` </details> **Still to test:** - Windows on AMD or Intel graphics, and Windows outside the market: an A/B run as above, expecting the closing `ab:` line with frames on both sides (by hand) ## Measurements **Verdict:** the A/B works and resolves the rule's cost in one run, scene by scene. On a Steam Deck OLED, side 1 (`=1`, the core given up) against side 0 (2, main's default when these runs were made): standing in Kokiri Forest 34.4 % less frame time, 19.1 against 29.1 fps (two runs); in the Castle Town market 15.7 % less, 14.7 against 17.4 fps (one run); in Hyrule Field no difference, 33.9 against 34.0 fps (two runs), because nothing is stream-decoded there and the switch was never read. On Windows with an RTX 4090 (Jiro113) the gap is far smaller and two runs don't settle it. This PR changes no default, so it has no before/after of its own: the table's columns are the A/B's two sides in the same runs. These runs were taken with this PR on main before https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1875, whose default was 2. On current main (3df5c13ed) the default on these PCs (a Steam Deck, an RTX 4090) is 1, so the same comparison is `HOS_AB=HOS_NATIVE_ZSTD=2` with the sides swapped; the A/B code is unchanged, and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1875 cites these numbers beside its own. Baseline: both columns are this PR's build (1b15e8a4d, v3.0.0-5-g1b15e8a4d) within the same runs: the before side is the rule in force, then main's default (`HOS_NATIVE_ZSTD` 2), the after side `HOS_NATIVE_ZSTD=1`. That is the comparison this PR exists for; main cannot alternate the rule. 1b15e8a4d is this PR's patch before the rebases (onto 868f6be87 and now 068fb8ebe), which moved only doc and comment lines: the code is the same. Setup: Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH (Mesa 26.2.0-devel 5c65d4fc31), AMD Custom APU 0932, 16 GB, GPU clock set by hand to 1600 MHz; Kokiri Forest standing, daytime; Hyrule Field after a teleport from Kokiri, 30 s standing then camera pans; the Castle Town market (child) after a teleport, standing, then one camera-pan cycle; 1280x800, Minimum, Render 50%, the game's own upscaler, sharpness 30, frame generation off, frame pacing on, no cap; headless, fresh options, no `longshot_env.txt`. `RX_VK_FRAME_TRACE=1 HOS_AB=HOS_NATIVE_ZSTD=1,5,150` (5 s periods after 150 s); five runs (Kokiri 2, Field 2, market 1), each about 300 s of A/B; `scripts/ab_summary.py` on the frame traces (both runs of a scene together, transition frames left out, bootstrap interval with seed 1) and each log's closing `ab:` line. To reproduce: a save standing in Kokiri Forest (or teleport to the market or Hyrule Field), the settings above, the two switches in `longshot_env.txt`, then `python -I scripts/ab_summary.py <game folder>/logs/longshot_frames.csv`. | | Before, `HOS_NATIVE_ZSTD` 2 (side 0, then the default): this PR @ 1b15e8a4d | After, `HOS_NATIVE_ZSTD=1` (side 1): this PR @ 1b15e8a4d | Δ | |---|---|---|---| | Kokiri Forest, both runs (120 periods): mean ms | 52.37 | 34.37 | -18.00 ms (-34.37 %), 95 % interval -18.78 to -17.19 | | Kokiri Forest: fps | 19.1 | 29.1 | +52.4 % | | Castle Town market, one run (57 periods): mean ms | 67.98 | 57.34 | -10.65 ms (-15.66 %), 95 % interval -13.17 to -8.05 | | Castle Town market: fps | 14.7 | 17.4 | +18.6 % | | Hyrule Field, both runs (74 periods): mean ms | 29.48 | 29.43 | -0.05 ms (-0.17 %), 95 % interval -0.69 to +0.61 (no difference) | | Hyrule Field: fps | 33.9 | 34.0 | +0.2 % | Each run's sides: | Run | Side 0 frames | Side 0 mean / median / p99 (ms) | Side 1 frames | Side 1 mean / median / p99 (ms) | |---|---|---|---|---| | Kokiri 1 | 2,767 | 52.72 / 51.87 / 72.19 | 4,115 | 34.28 / 33.29 / 52.07 | | Kokiri 2 | 2,711 | 52.01 / 51.26 / 70.98 | 3,955 | 34.46 / 33.61 / 49.41 | | Market 1 | 2,008 | 67.98 / 62.74 / 167.72 | 2,296 | 57.34 / 52.36 / 157.51 | | Field 1 | 2,983 | 29.92 / 28.86 / 53.46 | 2,883 | 29.34 / 28.00 / 51.71 | | Field 2 | 3,075 | 29.06 / 28.10 / 48.55 | 2,864 | 29.53 / 28.53 / 50.64 | The logs' closing lines (the host's own count, before `ab_summary.py` leaves out the periods' edges): ``` Kokiri 1: ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 2767 frames, 52.705 ms a frame (19.0 fps); side 1 (the A/B's value) 4121 frames, 34.278 ms a frame (29.2 fps); side 1 -34.96 % frame time; 438 frames in transitions left out Kokiri 2: ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 2712 frames, 52.014 ms a frame (19.2 fps); side 1 (the A/B's value) 3962 frames, 34.455 ms a frame (29.0 fps); side 1 -33.76 % frame time; 423 frames in transitions left out Market 1: ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 2014 frames, 68.101 ms a frame (14.7 fps); side 1 (the A/B's value) 2378 frames, 57.324 ms a frame (17.4 fps); side 1 -15.83 % frame time; 291 frames in transitions left out Field 1: ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 3059 frames, 29.880 ms a frame (33.5 fps); side 1 (the A/B's value) 3032 frames, 29.444 ms a frame (34.0 fps); side 1 -1.46 % frame time; 397 frames in transitions left out (...); HOS_NATIVE_ZSTD was never read during the A/B: no switch of that name takes part, both sides ran the same Field 2: ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 3075 frames, 29.057 ms a frame (34.4 fps); side 1 (the A/B's value) 3018 frames, 29.515 ms a frame (33.9 fps); side 1 +1.57 % frame time; 388 frames in transitions left out (...); HOS_NATIVE_ZSTD was never read during the A/B: no switch of that name takes part, both sides ran the same ``` The Field's "never read" note is the A/B working as it should: no stream decode happened there during the A/B, so both sides ran the same code and the difference is noise. The gap exists only while the game decodes in the background: another Steam Deck OLED saw none standing in Kokiri, with its decode threads idle (https://git.magicbean.dev/MagicBeanDev/Longshot/issues/1477). In the market run MemAvailable fell to 576 MiB at its lowest (one sample under 768 MiB), in a run both sides shared; in Kokiri and the Field it stayed above 1.6 GiB. All five runs exited 0 with no error lines. The Kokiri and the Field have two runs each; the market one run of 57 interleaved periods. **On Windows (Jiro113's runs, their numbers):** Windows 11, Intel Core i9-13900K, NVIDIA GeForce RTX 4090 (driver 621.24), 64 GB; this PR at c03bcb70 with the two commits of https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1729. Castle Town's south market (child), walking a scripted route through the control bridge, 2560x1440 windowed, HDR10, DLAA, frame generation off, V-Sync off, Reflex on, frame pacing on, no cap (`HOS_FPS_MAX=1000`); `RX_VK_FRAME_TRACE=1 HOS_AB=HOS_NATIVE_ZSTD=1,5,150`, two runs of about 315 s of A/B (126 periods together). Over both runs side 1 (`=1`) has 4.39 % less mean frame time, 65.7 against 62.8 fps, with a 95 % interval of -9.09 to +0.23 % that holds 0: the gap is far smaller than on the Deck and two runs don't settle it (run 2 alone -5.59 %, interval -10.81 to -0.91 %; run 1 -3.18 %, interval -11.70 to +4.50 %). The tail is shorter with 1 (p99 37.15 against 28.51 ms in run 1, 34.38 against 24.04 in run 2); the median is a little longer with 1 (14.31 against 14.66 ms over both). | Jiro113, Windows 11, RTX 4090, market walk, uncapped | Side 0: `HOS_NATIVE_ZSTD` 2 (then the default) | Side 1: `HOS_NATIVE_ZSTD` 1 | |---|---|---| | Run 1: frames | 9,511 | 9,524 | | Run 1: mean / median / p99 (ms) | 15.811 / 14.020 / 37.150 | 15.308 / 14.560 / 28.509 | | Run 2: frames | 9,373 | 9,621 | | Run 2: mean / median / p99 (ms) | 16.041 / 14.560 / 34.376 | 15.145 / 14.730 / 24.040 | | Both runs, 126 periods: mean (ms) | 15.925 | 15.226 | | Both runs: fps | 62.8 | 65.7 | | Side 1 less side 0 | | -0.699 ms (-4.39 %), 95 % interval -9.09 to +0.23 % | ``` ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 9507 frames, 15.811 ms a frame (63.2 fps); side 1 (the A/B's value) 9740 frames, 15.322 ms a frame (65.3 fps); side 1 -3.09 % frame time; 1245 frames in transitions left out (the host's presents; scripts/ab_summary.py on the frame trace for medians and a confidence interval) ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 9379 frames, 16.042 ms a frame (62.3 fps); side 1 (the A/B's value) 9792 frames, 15.163 ms a frame (66.0 fps); side 1 -5.48 % frame time; 1248 frames in transitions left out (the host's presents; scripts/ab_summary.py on the frame trace for medians and a confidence interval) ``` **The switch:** no new switch; `HOS_NATIVE_ZSTD` (2 keeps the virtual core, 1 gives it up, 0 the lifted decoder) ships at main's default, unchanged (unset: the GPU's, 1 except on Intel integrated GPUs, which keep 2; a value set wins), and is still read once without `HOS_AB`; with it, the A/B may alternate the mode in force with 1 or 2. Measured on a Steam Deck OLED (Kokiri, the market, Hyrule Field) and on Windows 11 with an RTX 4090 (the market, by Jiro113). Off (no `HOS_AB`), the added cost is one load a decode call; it was not timed apart. ## Risks A flip between two decode calls of one stream could break a decode; the test runs exactly that at every other call, and the rule touches nothing the stream keeps. Without `HOS_AB` nothing changes, so it is off unless a developer turns it on; a revert is enough to undo it. A player notices nothing after updating: saves, `longshot_options.ini` and the caches are untouched. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [x] Tested on my own PC, in the game for a change the game runs, as Testing says. [a Steam Deck OLED at 1b15e8a4d, the same code] - [x] `mise run test` passes (any failure that also fails on `main` is named). [at this head: the checks that read the changed files, the same as on main 3df5c13ed; the whole suite at the previous head, Testing] - [x] Measured against `main`, or Measurements says why there is nothing to measure. [the A/B's two sides in the same runs; main cannot alternate the rule: Baseline] - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A switch added or changed: it ships at the default the measurements chose, with its row in `gpu/vk/switches.def`. [`HOS_NATIVE_ZSTD`'s default, the GPU's from the device policy, unchanged; its row says what the A/B may flip] - [x] A check under `tests/` that reads changed source is updated, not weakened. - [x] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. [built and run on Windows by Jiro113 at c03bcb70; `host/hos_zstd.c` compiled for both at this head] - [x] Docs updated. [the `HOS_NATIVE_ZSTD` row in `gpu/vk/switches.def`] - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. [Windows on AMD or Intel graphics not run] - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). [the Steam Deck and an NVIDIA RTX 4090 on Windows; AMD RX 6000-9000 and Linux NVIDIA not run] - [ ] Approved in review, every comment answered. [no review yet] - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold. [the head is not pushed yet; the numbers are from 1b15e8a4d, the same A/B code, on main before https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1875 (default 2): Measurements says how they map onto the new default]
Fails until hos_zstd.c reads the switch through hos_ab_value at each call.

Assisted-by: Claude
feat(host): HOS_NATIVE_ZSTD in the in-run A/B
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
c03bcb70d7
Each stream decode call reads the core rule through hos_ab_value, so HOS_AB=HOS_NATIVE_ZSTD=1,3,60 alternates keeping the virtual core (2, the default) and giving it up (1) within one run. Only 1 and 2 flip: the core is given up and taken back within the call either way, and nothing else differs. 0 stays read once, since a stream begun on libzstd can't go on in the lifted decoder; an A/B value of 0 is taken as 2 and logged once.

Assisted-by: Claude
Author
Contributor

Tested in game: Steam Deck OLED, SteamOS 3.9.5, RADV (Mesa 26.2.0-devel): two Kokiri Forest runs with HOS_AB=HOS_NATIVE_ZSTD=1,5,150 at 1b15e8a4d (the same code before a rebase), both ended normally with both sides counted.

Tested in game: Steam Deck OLED, SteamOS 3.9.5, RADV (Mesa 26.2.0-devel): two Kokiri Forest runs with HOS_AB=HOS_NATIVE_ZSTD=1,5,150 at 1b15e8a4d (the same code before a rebase), both ended normally with both sides counted.
Contributor

Tested on Windows: it builds, and the A/B alternates and closes as Must be tested asks.

Build: Windows 11, llvm-mingw. This PR at c03bcb70, with #1729's two commits on top (19937d05, 44ab7f74), because main does not build on Windows without them since #475; this PR's host/ and nx2/ are unchanged by them. The checks this PR touches pass there, run one by one (not run_all.py): tests/test_native_zstd.py (with zstandard 0.25.0), tests/test_hos_ab.py, tests/test_switch_table.py, tests/test_doc_anchors.py, and #1729's tests/test_descriptor_array_census.py. In the game: that tree's longshot.exe and longshot_vk.dll (scripts/dev_packs.py), over the game data of a folder built at 544229a2.

Hardware: Intel Core i9-13900K, NVIDIA GeForce RTX 4090 (driver 621.24), 64 GB, Windows 11

Must be tested: HOS_AB=HOS_NATIVE_ZSTD=1,3,60 and RX_VK_FRAME_TRACE=1, a save walking in Castle Town's south market for three minutes, driven and closed through the control bridge (its quit: the game's own close, which prints the line). The log's last ab: line, frames on both sides and no "never read" note (and no "taken as 2" line):

ab: HOS_NATIVE_ZSTD=1 against its normal value, 3.0 s periods after 60 s: side 0 (normal) 4606 frames, 18.281 ms a frame (54.7 fps); side 1 (the A/B's value) 4963 frames, 16.872 ms a frame (59.3 fps); side 1 -7.71 % frame time; 1091 frames in transitions left out (the host's presents; scripts/ab_summary.py on the frame trace for medians and a confidence interval)

The frame trace has the ab_side column and scripts/ab_summary.py reads it (61 periods; side 1 -7.65 %, 95 % interval -11.08 to -4.64 %). That run had DLSS Frame Generation 2x with V-Sync Auto (FIFO on a 120 Hz screen), which holds the game to 60 frames a second: side 1 sits at that ceiling (median 16.64 ms), side 0 just above it (median 17.06 ms, 57.4 % of its frames at 16.7 ms or more against 48.7 %), and its p99 is 37.59 ms against 26.17.

Uncapped, as in the Deck runs: the tail is shorter with 1 in both runs, p99 37.15 against 28.51 ms and 34.38 against 24.04 (ab_summary.py gives no interval for p99), but the mean moves much less than on the Deck (a different scene too: walking in the Market, not standing in Kokiri), and two runs don't settle it. Over both runs side 1 has 4.4 % less mean frame time (62.8 against 65.7 fps), with a 95 % interval of -9.09 to +0.23 % that holds 0; run 2 alone shows a gap (-5.59 %, interval -10.81 to -0.91 %), run 1 doesn't (-3.18 %, interval -11.70 to +4.50 %). The median is a little longer with 1 (14.31 against 14.66 ms over both). Same PC and scene as the capped, profiled cell in #1477 (comment); these runs have no profiling switches.

Scene: Castle Town's south market (child), walking a scripted route through the control bridge; 2560x1440 in a window, HDR10, DLAA, frame generation off, V-Sync off, Reflex on, frame pacing on, no cap (HOS_FPS_MAX=1000)
Method: RX_VK_FRAME_TRACE=1 HOS_AB=HOS_NATIVE_ZSTD=1,5,150, about 315 s of A/B a run (63 periods), two runs; scripts/ab_summary.py on both frame traces (the game's own frames); the build above

Side 0: HOS_NATIVE_ZSTD 2 (default), c03bcb70 + #1729 Side 1: HOS_NATIVE_ZSTD 1, c03bcb70 + #1729
Run 1: frames 9,511 9,524
Run 1: mean / median / p99 (ms) 15.811 / 14.020 / 37.150 15.308 / 14.560 / 28.509
Run 2: frames 9,373 9,621
Run 2: mean / median / p99 (ms) 16.041 / 14.560 / 34.376 15.145 / 14.730 / 24.040
Both runs, 126 periods: mean (ms) 15.925 15.226
Both runs: fps 62.8 65.7

Side 1 less side 0, both runs: -0.699 ms (-4.39 %), 95 % interval -1.447 to +0.037 ms (-9.09 to +0.23 %). The table's frames are ab_summary.py's: the frame trace's, without the last, unfinished period; the closing lines count the host's presents.

ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 9507 frames, 15.811 ms a frame (63.2 fps); side 1 (the A/B's value) 9740 frames, 15.322 ms a frame (65.3 fps); side 1 -3.09 % frame time; 1245 frames in transitions left out (the host's presents; scripts/ab_summary.py on the frame trace for medians and a confidence interval)
ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 9379 frames, 16.042 ms a frame (62.3 fps); side 1 (the A/B's value) 9792 frames, 15.163 ms a frame (66.0 fps); side 1 -5.48 % frame time; 1248 frames in transitions left out (the host's presents; scripts/ab_summary.py on the frame trace for medians and a confidence interval)

If a docs/tested/ file for these runs would help, I can write its text.

Tested on Windows: it builds, and the A/B alternates and closes as Must be tested asks. **Build:** Windows 11, llvm-mingw. This PR at c03bcb70, with #1729's two commits on top (19937d05, 44ab7f74), because main does not build on Windows without them since #475; this PR's `host/` and `nx2/` are unchanged by them. The checks this PR touches pass there, run one by one (not `run_all.py`): `tests/test_native_zstd.py` (with `zstandard` 0.25.0), `tests/test_hos_ab.py`, `tests/test_switch_table.py`, `tests/test_doc_anchors.py`, and #1729's `tests/test_descriptor_array_census.py`. In the game: that tree's `longshot.exe` and `longshot_vk.dll` (`scripts/dev_packs.py`), over the game data of a folder built at 544229a2. Hardware: Intel Core i9-13900K, NVIDIA GeForce RTX 4090 (driver 621.24), 64 GB, Windows 11 **Must be tested:** `HOS_AB=HOS_NATIVE_ZSTD=1,3,60` and `RX_VK_FRAME_TRACE=1`, a save walking in Castle Town's south market for three minutes, driven and closed through the control bridge (its `quit`: the game's own close, which prints the line). The log's last `ab:` line, frames on both sides and no "never read" note (and no "taken as 2" line): ``` ab: HOS_NATIVE_ZSTD=1 against its normal value, 3.0 s periods after 60 s: side 0 (normal) 4606 frames, 18.281 ms a frame (54.7 fps); side 1 (the A/B's value) 4963 frames, 16.872 ms a frame (59.3 fps); side 1 -7.71 % frame time; 1091 frames in transitions left out (the host's presents; scripts/ab_summary.py on the frame trace for medians and a confidence interval) ``` The frame trace has the `ab_side` column and `scripts/ab_summary.py` reads it (61 periods; side 1 -7.65 %, 95 % interval -11.08 to -4.64 %). That run had DLSS Frame Generation 2x with V-Sync Auto (FIFO on a 120 Hz screen), which holds the game to 60 frames a second: side 1 sits at that ceiling (median 16.64 ms), side 0 just above it (median 17.06 ms, 57.4 % of its frames at 16.7 ms or more against 48.7 %), and its p99 is 37.59 ms against 26.17. **Uncapped, as in the Deck runs:** the tail is shorter with 1 in both runs, p99 37.15 against 28.51 ms and 34.38 against 24.04 (`ab_summary.py` gives no interval for p99), but the mean moves much less than on the Deck (a different scene too: walking in the Market, not standing in Kokiri), and two runs don't settle it. Over both runs side 1 has 4.4 % less mean frame time (62.8 against 65.7 fps), with a 95 % interval of -9.09 to +0.23 % that holds 0; run 2 alone shows a gap (-5.59 %, interval -10.81 to -0.91 %), run 1 doesn't (-3.18 %, interval -11.70 to +4.50 %). The median is a little longer with 1 (14.31 against 14.66 ms over both). Same PC and scene as the capped, profiled cell in https://git.magicbean.dev/MagicBeanDev/Longshot/issues/1477#issuecomment-53913; these runs have no profiling switches. Scene: Castle Town's south market (child), walking a scripted route through the control bridge; 2560x1440 in a window, HDR10, DLAA, frame generation off, V-Sync off, Reflex on, frame pacing on, no cap (`HOS_FPS_MAX=1000`) Method: `RX_VK_FRAME_TRACE=1 HOS_AB=HOS_NATIVE_ZSTD=1,5,150`, about 315 s of A/B a run (63 periods), two runs; `scripts/ab_summary.py` on both frame traces (the game's own frames); the build above | | Side 0: `HOS_NATIVE_ZSTD` 2 (default), c03bcb70 + #1729 | Side 1: `HOS_NATIVE_ZSTD` 1, c03bcb70 + #1729 | |---|---|---| | Run 1: frames | 9,511 | 9,524 | | Run 1: mean / median / p99 (ms) | 15.811 / 14.020 / 37.150 | 15.308 / 14.560 / 28.509 | | Run 2: frames | 9,373 | 9,621 | | Run 2: mean / median / p99 (ms) | 16.041 / 14.560 / 34.376 | 15.145 / 14.730 / 24.040 | | Both runs, 126 periods: mean (ms) | 15.925 | 15.226 | | Both runs: fps | 62.8 | 65.7 | Side 1 less side 0, both runs: -0.699 ms (-4.39 %), 95 % interval -1.447 to +0.037 ms (-9.09 to +0.23 %). The table's frames are `ab_summary.py`'s: the frame trace's, without the last, unfinished period; the closing lines count the host's presents. ``` ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 9507 frames, 15.811 ms a frame (63.2 fps); side 1 (the A/B's value) 9740 frames, 15.322 ms a frame (65.3 fps); side 1 -3.09 % frame time; 1245 frames in transitions left out (the host's presents; scripts/ab_summary.py on the frame trace for medians and a confidence interval) ab: HOS_NATIVE_ZSTD=1 against its normal value, 5.0 s periods after 150 s: side 0 (normal) 9379 frames, 16.042 ms a frame (62.3 fps); side 1 (the A/B's value) 9792 frames, 15.163 ms a frame (66.0 fps); side 1 -5.48 % frame time; 1248 frames in transitions left out (the host's presents; scripts/ab_summary.py on the frame trace for medians and a confidence interval) ``` If a `docs/tested/` file for these runs would help, I can write its text.
docs(tested): Steam Deck OLED in-run A/Bs of the zstd core rule
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 3s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
6edf79347f
Assisted-by: Claude
Author
Contributor

Measurements now include the Deck's other two scenes. In the Castle Town market, =1 gives 17.4 fps against 14.7 at the default (-10.65 ms, 95 % interval -13.17 to -8.05, 57 periods). In Hyrule Field the two sides match (34.0 against 33.9 fps, interval -0.69 to +0.61, 74 periods), because nothing is stream-decoded there and the log says the switch was never read. Jiro113's Windows build and run (#1726, 2026-11-19 13:58) settles Must be tested and the Windows box; the new commit adds one docs/tested/ file per Deck run (five), and run_all's 49 failures are named, all of them the same on main. Thank you @Jiro113 for the Windows build and runs.

Measurements now include the Deck's other two scenes. In the Castle Town market, `=1` gives 17.4 fps against 14.7 at the default (-10.65 ms, 95 % interval -13.17 to -8.05, 57 periods). In Hyrule Field the two sides match (34.0 against 33.9 fps, interval -0.69 to +0.61, 74 periods), because nothing is stream-decoded there and the log says the switch was never read. Jiro113's Windows build and run (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1726, 2026-11-19 13:58) settles Must be tested and the Windows box; the new commit adds one `docs/tested/` file per Deck run (five), and run_all's 49 failures are named, all of them the same on `main`. Thank you @Jiro113 for the Windows build and runs.
zeldaootr changed title from WIP: feat(host): HOS_NATIVE_ZSTD in the in-run A/B to feat(host): HOS_NATIVE_ZSTD in the in-run A/B 2026-11-19 14:57:56 +00:00
zeldaootr force-pushed ab-native-zstd from 6edf79347f
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 3s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 88ff712398
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-20 03:40:04 +00:00
Compare
Author
Contributor

Rebased onto main 068fb8ebe, the docs and tools revamp (head 88ff71239, both code commits kept). Main no longer lists the in-run A/B's switches in prose: docs/dev/developing.md points to their switches.def rows, so this PR's doc text is now the HOS_NATIVE_ZSTD row in gpu/vk/switches.def (1 and 2 may be flipped, 0 may not), and hos_zstd.c drops its docs/tested/ reference. Main removed the tested-configurations catalogue, so the commit with the five docs/tested/ run files is dropped and their numbers live in the description's Measurements, which now follows the new template. The game-free checks fail nothing that main doesn't also fail on this PC, and tests/test_native_zstd.py, run with the zstandard module, passes.

Rebased onto main 068fb8ebe, the docs and tools revamp (head 88ff71239, both code commits kept). Main no longer lists the in-run A/B's switches in prose: `docs/dev/developing.md` points to their `switches.def` rows, so this PR's doc text is now the `HOS_NATIVE_ZSTD` row in `gpu/vk/switches.def` (1 and 2 may be flipped, 0 may not), and `hos_zstd.c` drops its `docs/tested/` reference. Main removed the tested-configurations catalogue, so the commit with the five `docs/tested/` run files is dropped and their numbers live in the description's Measurements, which now follows the new template. The game-free checks fail nothing that main doesn't also fail on this PC, and `tests/test_native_zstd.py`, run with the `zstandard` module, passes.
Author
Contributor

One thing for a reviewer to weigh, now that main has dev.py ab: because this PR wires HOS_NATIVE_ZSTD into the in-run A/B, dev.py ab --env HOS_NATIVE_ZSTD=0 would pick an in-run A/B for it, and the value 0 cannot be flipped in a run (a stream begun on libzstd can't go on in the lifted decoder), so zs_core takes it as 2 and logs that once. Both sides would then run the same code and the write-up would read "within the spread". Today only the switch's row in gpu/vk/switches.def warns of it ("HOS_AB may flip it between 1 and 2, not to 0 (read once: dev.py ab --separate)").

I have not changed scripts/devtools/ab.py here, since that is another topic and another owner's file. Two ways it could be closed, whichever you prefer: dev.py ab refusing an in-run A/B when the game logs that it took the value as another (the log line is there to key on), or a per-switch list of values the in-run A/B accepts next to tests/test_hos_ab.py's WIRED list. Say which and I'll do it in a follow-up.

One thing for a reviewer to weigh, now that main has `dev.py ab`: because this PR wires `HOS_NATIVE_ZSTD` into the in-run A/B, `dev.py ab --env HOS_NATIVE_ZSTD=0` would pick an in-run A/B for it, and the value 0 cannot be flipped in a run (a stream begun on libzstd can't go on in the lifted decoder), so `zs_core` takes it as 2 and logs that once. Both sides would then run the same code and the write-up would read "within the spread". Today only the switch's row in `gpu/vk/switches.def` warns of it ("HOS_AB may flip it between 1 and 2, not to 0 (read once: dev.py ab --separate)"). I have not changed `scripts/devtools/ab.py` here, since that is another topic and another owner's file. Two ways it could be closed, whichever you prefer: `dev.py ab` refusing an in-run A/B when the game logs that it took the value as another (the log line is there to key on), or a per-switch list of values the in-run A/B accepts next to `tests/test_hos_ab.py`'s WIRED list. Say which and I'll do it in a follow-up.
Contributor

Same picture on a Ryzen desktop: HOS_NATIVE_ZSTD=1 cuts the Market's slow frames by about a third. Would you take the default to 1 per platform in this PR, or do you mind if I open a PR for that which builds on this one?

Ryzen 9 9900X, RTX 4070 SUPER, Windows 11, main 89c3d21e; a recorded Market walk with camera turns, 4K DLSS Quality:

Market =2 (2 runs) =1 (3 runs)
frames over 20 ms 42.5 26
frames over 25 ms 10.5 2
fps 71.8 72.3
1 % low 42.2 fps 46.9 fps

No hang at a load in ~20 runs with =1. Since #1632's Core Ultra 7 155H hasn't been checked after #1553 yet, I'd suggest =1 everywhere except Intel iGPUs for now.

**Same picture on a Ryzen desktop: `HOS_NATIVE_ZSTD=1` cuts the Market's slow frames by about a third. Would you take the default to 1 per platform in this PR, or do you mind if I open a PR for that which builds on this one?** Ryzen 9 9900X, RTX 4070 SUPER, Windows 11, main 89c3d21e; a recorded Market walk with camera turns, 4K DLSS Quality: | Market | `=2` (2 runs) | `=1` (3 runs) | |---|---|---| | frames over 20 ms | 42.5 | 26 | | frames over 25 ms | 10.5 | 2 | | fps | 71.8 | 72.3 | | 1 % low | 42.2 fps | 46.9 fps | No hang at a load in ~20 runs with `=1`. Since #1632's Core Ultra 7 155H hasn't been checked after #1553 yet, I'd suggest `=1` everywhere except Intel iGPUs for now.
Author
Contributor

Thanks, that matches the Steam Deck. Please go ahead and open the default change as its own PR on top of this one: this PR only adds the switch to the in-run A/B, and I would rather keep it to that so it can merge without waiting on the per-platform decision.

For your PR, the Deck numbers from #1477 (Steam Deck OLED, v3.0.0, Minimum, Render 50%, uncapped, in-run A/B): Kokiri Forest 19.1 fps at =2 against 29.1 at =1 (-18.00 ms a frame, CI -18.78 to -17.19), the Market 14.7 against 17.4, Hyrule Field no difference (the switch is never read there). So =1 on the Deck is supported by what I measured; I have no Intel iGPU to say anything about that exception, and no hang at a load in my runs with =1 either. I can run your PR on the Deck when it is up.

Thanks, that matches the Steam Deck. Please go ahead and open the default change as its own PR on top of this one: this PR only adds the switch to the in-run A/B, and I would rather keep it to that so it can merge without waiting on the per-platform decision. For your PR, the Deck numbers from https://git.magicbean.dev/MagicBeanDev/Longshot/issues/1477 (Steam Deck OLED, v3.0.0, Minimum, Render 50%, uncapped, in-run A/B): Kokiri Forest 19.1 fps at `=2` against 29.1 at `=1` (-18.00 ms a frame, CI -18.78 to -17.19), the Market 14.7 against 17.4, Hyrule Field no difference (the switch is never read there). So `=1` on the Deck is supported by what I measured; I have no Intel iGPU to say anything about that exception, and no hang at a load in my runs with `=1` either. I can run your PR on the Deck when it is up.
Contributor

Opened it as #1875: HOS_NATIVE_ZSTD defaults to 1 everywhere except Intel iGPUs, which keep 2.

If you'd like to run it on the Deck, a test in Kokiri Forest and the market against main would show whether your gain holds with the per-GPU default; the log should say HOS_NATIVE_ZSTD=1, this GPU's default.

**Opened it as #1875: `HOS_NATIVE_ZSTD` defaults to 1 everywhere except Intel iGPUs, which keep 2.** If you'd like to run it on the Deck, a test in Kokiri Forest and the market against main would show whether your gain holds with the per-GPU default; the log should say `HOS_NATIVE_ZSTD=1, this GPU's default`.
zeldaootr force-pushed ab-native-zstd from 88ff712398
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 7c5cf948a7
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 28s
2026-11-20 22:42:15 +00:00
Compare
Author
Contributor

Rebased onto main 3df5c13ed, after #1875 made the default the GPU's (1 except on Intel integrated GPUs) while the switch is unset: the A/B now flips whichever mode is in force, set or the GPU's default, as the other wired switches do, and the test adds those unset cases (head 7c5cf948a, test commit cbd3c4389). The checks that read the changed files and the Linux and Windows compiles match main; the in-game numbers are from before the rebase, when the default was 2, and the description now says how they map onto the new default.

Rebased onto main 3df5c13ed, after https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1875 made the default the GPU's (1 except on Intel integrated GPUs) while the switch is unset: the A/B now flips whichever mode is in force, set or the GPU's default, as the other wired switches do, and the test adds those unset cases (head 7c5cf948a, test commit cbd3c4389). The checks that read the changed files and the Linux and Windows compiles match main; the in-game numbers are from before the rebase, when the default was 2, and the description now says how they map onto the new default.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 28s
This pull request has changes conflicting with the target branch.
  • tests/test_hos_ab.py
Some workflows are waiting to be reviewed.
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'ab-native-zstd:zeldaootr-ab-native-zstd'
git switch 'zeldaootr-ab-native-zstd'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1726
No description provided.