WIP: fix(host): native zstd keeps the virtual core during stream decodes (load freezes on an Arc iGPU) #1632

Draft
Guep wants to merge 1 commit from Guep/Longshot:fix/zstd-keep-core into main
Contributor

Type: fix · Risk: medium · Tested on: Windows 11, Arc 155H iGPU (Intel) · Needs: Windows on another iGPU, a run with #1553

TL;DR

On an Intel Arc 155H laptop, main often froze at loading screens for 20 seconds to 5 minutes: 7 runs of 10 on a fixed route. Each freeze followed the native zstd stream decode giving up its virtual core and waiting to get it back. Keeping the core during that decode (HOS_NATIVE_ZSTD=2) froze in none of 3 runs, and earlier tours on an RTX 4070 SUPER showed loading screens no longer with it. This PR makes keeping the core the default; HOS_NATIVE_ZSTD=1 gives it up as before.

Breakdown

  • Who it helps: players whose game freezes for seconds or minutes when an area loads (seen on an Arc 155H iGPU).
  • What was wrong: the stream decode gives its virtual core up and takes it back around every call; on this PC a load then stalls while guest threads wait seconds for a core.
  • What changes: the stream decode keeps its virtual core while libzstd runs (what HOS_NATIVE_ZSTD=2 did).
  • What does not change: decoding on the host's libzstd, the whole-buffer decode (HOS_NATIVE_ZSTD_FRAMES), HOS_NATIVE_ZSTD=0.
  • How it was checked: the same route on main with each setting, 3 to 10 runs each; the native zstd game-free test.
  • What is still open: a run on a build with #1553 (it changed virtual-core wakeups after these runs), and other PCs.
  • Risk: medium: the decode holds a core for its call again, as before #1402's give-up, which #1402 meant to free for other threads.

Why

Since #1402 made the native stream decode the default with its core given up meanwhile, main froze at loads on this PC in 7 runs of 10: the game reads no controller for 20 s to 5 min, always at the same place (hid: the game read no controller for N s (lr 0x83489670)). With HOS_NATIVE_ZSTD=0 HOS_NATIVE_ZSTD_FRAMES=0 it froze in none of 3, and with HOS_NATIVE_ZSTD=2 (on the host, the core kept) in none of 3: the give-up and take-back are what make the freezes frequent here. The same log line is in reports from other players: See #1424, See #1365. A rare freeze at the same place also happened once on v3.0.0-rc.7 (before #1402), so this is not the only cause.

How to reproduce

Seen on: Windows 11 Pro, Arc 155H iGPU (0x7D55), driver 101.9175, Core Ultra 7 155H, 32 GB
Settings and switches: the game's upscaler 50%, FSR 3.1 FG 2x; RX_VK_EQ_BIAS=0; native zstd at its default
Steps: a scripted route by teleport: Kokiri Forest, Hyrule Field, Market, Kakariko, Death Mountain Trail (about 18 minutes)
Expected: each area loads within seconds
Actual: in most runs one load hangs for 20 s to 5 min; the log shows hid: the game read no controller with lr 0x83489670

What changed

The default of HOS_NATIVE_ZSTD (unset or empty) is now 2: the stream decode runs on the host's libzstd and keeps its virtual core. HOS_NATIVE_ZSTD=1 still gives it up around each call.

  • host: zs_mode() in host/hos_zstd.c (the default 2; 1 only when asked), and its comment.
  • tests: tests/test_native_zstd.py expects the core kept when the switch is unset or empty.
  • docs: docs/development.md and gpu/vk/switches.def describe the new default.

Measurements

Verdict: freezes at loads 7 of 10 runs with the core given up, 0 of 3 with it kept, on main c7db6c02 / 257c5491.

Hardware: Windows 11 Pro, Arc 155H iGPU, driver 101.9175, Core Ultra 7 155H, 32 GB
Scene: the five-area route above; the game's upscaler 50%, FSR 3.1 FG 2x
Method: a run counts as frozen when longshot_log.txt has hid: the game read no controller for N s with N over 20; runs interleaved over one night
How to reproduce: the route with HOS_NATIVE_ZSTD unset, then =2

Before: main @ c7db6c02 / 257c5491, native zstd default (=1) HOS_NATIVE_ZSTD=0, HOS_NATIVE_ZSTD_FRAMES=0 After: HOS_NATIVE_ZSTD=2 (this PR's default), main @ c7db6c02
Runs frozen at a load (over 20 s) 7 of 10 0 of 3 0 of 3

The RTX 4070 SUPER tours in docs/tested/2026-11-17-native-zstd-default-* (keep vs give up) show the same loading-screen time either way.

Flags and settings

HOS_NATIVE_ZSTD's default changes from 1 to 2.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
HOS_NATIVE_ZSTD the game's zstd stream decode on the host unset = 1 (core given up) unset = 2 (core kept) the decode keeps its virtual core helps on an Arc 155H (no freezes); same load times on an RTX 4070 SUPER 2 for everyone; 1 only for testing

Testing

Tested: Windows 11, Arc 155H: the route in the game with HOS_NATIVE_ZSTD=2 on main c7db6c02, 3 runs, no freeze (this PR makes that the default; the build of this exact commit was not run in the game). tests/test_native_zstd.py passes.

Not tested: this commit's own build in the game, and a build with #1553 (it reworked virtual-core wakeups after these runs and may change the picture): a run on the nightly with #1553 is under way on this PC and its result will be posted here. Linux and other GPUs.

Must be tested:

  • Windows, an Intel iGPU: a teleport tour of five areas, expecting no load to hang (no the game read no controller for over 20 s in longshot_log.txt) (Test Farm: @Beanstalk windows intel, a teleport tour, look for the hid line)
  • any PC: loading-screen time on a teleport tour, expecting it unchanged (by hand: docs/tested/2026-11-17-native-zstd-default-*'s method)

Risks

What could break: loads could be slightly slower where other game threads needed the core the decode now keeps.

Turning it off: HOS_NATIVE_ZSTD=1 (the previous default).

Compatibility: nothing changes (no cache, save or option change).

Issues

See #1424
See #1365

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs: the default it sets was run in the game (HOS_NATIVE_ZSTD=2); this commit's build not yet.
  • Measurements has a verdict.
  • Nothing gets worse that I know of.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (the 39 tests that fail on this PC (Windows, no Vulkan headers or spirv-val here) fail on main c507b98c too; none new).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched.
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Switches documented (docs/development.md) and shipped at their recommended value.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.
**Type:** fix · **Risk:** medium · **Tested on:** Windows 11, Arc 155H iGPU (Intel) · **Needs:** Windows on another iGPU, a run with #1553 ## TL;DR On an Intel Arc 155H laptop, main often froze at loading screens for 20 seconds to 5 minutes: 7 runs of 10 on a fixed route. Each freeze followed the native zstd stream decode giving up its virtual core and waiting to get it back. Keeping the core during that decode (`HOS_NATIVE_ZSTD=2`) froze in none of 3 runs, and earlier tours on an RTX 4070 SUPER showed loading screens no longer with it. This PR makes keeping the core the default; `HOS_NATIVE_ZSTD=1` gives it up as before. ## Breakdown - **Who it helps:** players whose game freezes for seconds or minutes when an area loads (seen on an Arc 155H iGPU). - **What was wrong:** the stream decode gives its virtual core up and takes it back around every call; on this PC a load then stalls while guest threads wait seconds for a core. - **What changes:** the stream decode keeps its virtual core while libzstd runs (what `HOS_NATIVE_ZSTD=2` did). - **What does not change:** decoding on the host's libzstd, the whole-buffer decode (`HOS_NATIVE_ZSTD_FRAMES`), `HOS_NATIVE_ZSTD=0`. - **How it was checked:** the same route on main with each setting, 3 to 10 runs each; the native zstd game-free test. - **What is still open:** a run on a build with #1553 (it changed virtual-core wakeups after these runs), and other PCs. - **Risk:** medium: the decode holds a core for its call again, as before #1402's give-up, which #1402 meant to free for other threads. ## Why Since #1402 made the native stream decode the default with its core given up meanwhile, main froze at loads on this PC in 7 runs of 10: the game reads no controller for 20 s to 5 min, always at the same place (`hid: the game read no controller for N s (lr 0x83489670)`). With `HOS_NATIVE_ZSTD=0 HOS_NATIVE_ZSTD_FRAMES=0` it froze in none of 3, and with `HOS_NATIVE_ZSTD=2` (on the host, the core kept) in none of 3: the give-up and take-back are what make the freezes frequent here. The same log line is in reports from other players: See #1424, See #1365. A rare freeze at the same place also happened once on v3.0.0-rc.7 (before #1402), so this is not the only cause. ## How to reproduce **Seen on:** Windows 11 Pro, Arc 155H iGPU (0x7D55), driver 101.9175, Core Ultra 7 155H, 32 GB **Settings and switches:** the game's upscaler 50%, FSR 3.1 FG 2x; `RX_VK_EQ_BIAS=0`; native zstd at its default **Steps:** a scripted route by teleport: Kokiri Forest, Hyrule Field, Market, Kakariko, Death Mountain Trail (about 18 minutes) **Expected:** each area loads within seconds **Actual:** in most runs one load hangs for 20 s to 5 min; the log shows `hid: the game read no controller` with lr 0x83489670 ## What changed The default of `HOS_NATIVE_ZSTD` (unset or empty) is now 2: the stream decode runs on the host's libzstd and keeps its virtual core. `HOS_NATIVE_ZSTD=1` still gives it up around each call. - host: `zs_mode()` in `host/hos_zstd.c` (the default 2; 1 only when asked), and its comment. - tests: `tests/test_native_zstd.py` expects the core kept when the switch is unset or empty. - docs: `docs/development.md` and `gpu/vk/switches.def` describe the new default. ## Measurements **Verdict:** freezes at loads 7 of 10 runs with the core given up, 0 of 3 with it kept, on main c7db6c02 / 257c5491. Hardware: Windows 11 Pro, Arc 155H iGPU, driver 101.9175, Core Ultra 7 155H, 32 GB Scene: the five-area route above; the game's upscaler 50%, FSR 3.1 FG 2x Method: a run counts as frozen when longshot_log.txt has `hid: the game read no controller for N s` with N over 20; runs interleaved over one night How to reproduce: the route with `HOS_NATIVE_ZSTD` unset, then `=2` | | Before: main @ c7db6c02 / 257c5491, native zstd default (=1) | `HOS_NATIVE_ZSTD=0`, `HOS_NATIVE_ZSTD_FRAMES=0` | After: `HOS_NATIVE_ZSTD=2` (this PR's default), main @ c7db6c02 | |---|---|---|---| | Runs frozen at a load (over 20 s) | 7 of 10 | 0 of 3 | 0 of 3 | The RTX 4070 SUPER tours in `docs/tested/2026-11-17-native-zstd-default-*` (keep vs give up) show the same loading-screen time either way. ### Flags and settings `HOS_NATIVE_ZSTD`'s default changes from 1 to 2. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `HOS_NATIVE_ZSTD` | the game's zstd stream decode on the host | unset = 1 (core given up) | unset = 2 (core kept) | the decode keeps its virtual core | helps on an Arc 155H (no freezes); same load times on an RTX 4070 SUPER | 2 for everyone; 1 only for testing | ## Testing **Tested:** Windows 11, Arc 155H: the route in the game with `HOS_NATIVE_ZSTD=2` on main c7db6c02, 3 runs, no freeze (this PR makes that the default; the build of this exact commit was not run in the game). `tests/test_native_zstd.py` passes. **Not tested:** this commit's own build in the game, and a build with #1553 (it reworked virtual-core wakeups after these runs and may change the picture): a run on the nightly with #1553 is under way on this PC and its result will be posted here. Linux and other GPUs. **Must be tested:** - Windows, an Intel iGPU: a teleport tour of five areas, expecting no load to hang (no `the game read no controller for` over 20 s in longshot_log.txt) (Test Farm: @Beanstalk windows intel, a teleport tour, look for the hid line) - any PC: loading-screen time on a teleport tour, expecting it unchanged (by hand: `docs/tested/2026-11-17-native-zstd-default-*`'s method) ## Risks **What could break:** loads could be slightly slower where other game threads needed the core the decode now keeps. **Turning it off:** `HOS_NATIVE_ZSTD=1` (the previous default). **Compatibility:** nothing changes (no cache, save or option change). ## Issues See #1424 See #1365 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [ ] Tested on my own PC where it runs: the default it sets was run in the game (`HOS_NATIVE_ZSTD=2`); this commit's build not yet. - [x] Measurements has a verdict. - [x] Nothing gets worse that I know of. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (the 39 tests that fail on this PC (Windows, no Vulkan headers or spirv-val here) fail on main c507b98c too; none new). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched. - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Switches documented (docs/development.md) and shipped at their recommended value. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated.
fix(host): native zstd keeps the virtual core during stream decodes (load freezes on an Arc iGPU)
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 5s
Website / Website on Cloudflare (pull_request_target) Has been skipped
ab2d06597e
With the core given up and taken back around each stream decode
(HOS_NATIVE_ZSTD=1, the default since #1402), main froze at loads in 7
runs of 10 on an Arc 155H iGPU: the game read no controller for 20 s to
5 min (lr 0x83489670). Keeping the core (=2) froze in none of 3, and the
RTX 4070 SUPER tours in docs/tested show loading screens no longer with
it. The default is now 2; HOS_NATIVE_ZSTD=1 gives the core up as before.

Assisted-by: Claude Code
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_014iDJjN9rPpzX7TDZDbxAwN
Author
Contributor

Tested in game: Windows 11, Intel Arc 155H iGPU (0x7D55), driver 101.9175: a five-area teleport route (Kokiri, Hyrule Field, Market, Kakariko, Death Mountain Trail) on main c7db6c02 with HOS_NATIVE_ZSTD=2, the default this PR sets: 3 runs, no load hung (no 'the game read no controller' over 20 s); with the previous default main hung at a load in 7 runs of 10. This commit's own build not yet run in the game.

Tested in game: Windows 11, Intel Arc 155H iGPU (0x7D55), driver 101.9175: a five-area teleport route (Kokiri, Hyrule Field, Market, Kakariko, Death Mountain Trail) on main c7db6c02 with HOS_NATIVE_ZSTD=2, the default this PR sets: 3 runs, no load hung (no 'the game read no controller' over 20 s); with the previous default main hung at a load in 7 runs of 10. This commit's own build not yet run in the game.
Collaborator

Possibly related, for context:

  • issue #539 Game: freeze requiring hard reboot — Open area-transition freeze report with unconfirmed cause; similar symptom.
Possibly related, for context: - issue #539 Game: freeze requiring hard reboot — Open area-transition freeze report with unconfirmed cause; similar symptom.
Author
Contributor

Thanks. I read #539 and its log: it looks like a different problem.

  • #539 froze the whole PC (hard reboot); here only the game hangs, for 20 s to 5 min, and then goes on or closes normally.
  • Its log is v3.0.0-rc.3 (source cb01b5a3) with no zstd: line, so the native stream decode this PR changes was not running there.
  • It has no hid: the game read no controller for N s (lr 0x83489670) line, which every freeze here has, as do #1424 and #1365.

What it shares is virtual-core starvation before the end (vcores: ... longest 1926.3 ms on several threads), the same family as #1553 works on, but not this decode's give-up and take-back. So I'm leaving #539 out of this PR's Issues.

Thanks. I read #539 and its log: it looks like a different problem. - #539 froze the whole PC (hard reboot); here only the game hangs, for 20 s to 5 min, and then goes on or closes normally. - Its log is v3.0.0-rc.3 (source cb01b5a3) with no `zstd:` line, so the native stream decode this PR changes was not running there. - It has no `hid: the game read no controller for N s (lr 0x83489670)` line, which every freeze here has, as do #1424 and #1365. What it shares is virtual-core starvation before the end (`vcores: ... longest 1926.3 ms` on several threads), the same family as #1553 works on, but not this decode's give-up and take-back. So I'm leaving #539 out of this PR's Issues.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 5s
Website / Website on Cloudflare (pull_request_target) Has been skipped
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/zstd-keep-core:Guep-fix/zstd-keep-core'
git switch 'Guep-fix/zstd-keep-core'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1632
No description provided.