fix(host): a SARC name offset is 24 bits, not 16 #2145

Merged
HumanBean merged 1 commit from dazemc/Longshot:fix/sarc-name-offset-24bit into main 2026-11-21 22:44:58 +00:00
Contributor

Summary

heart_sarc_find reads a SARC SFAT node's name offset as attributes & 0xffff. In the format, bits 0-23 are the name's word offset into the SFNT string table and bit 24 is the has-name flag (already tested on the line above). Masking with 16 bits drops bits 16-23, so in a string table of 256 KiB or more every name past that point would resolve at the wrong offset and be looked up as absent.

This is latent, not a game bug: the shipped Pack/ZsDic.pack.zs — the only file this reads — has a 56-byte name table whose largest name word offset is 0xb, so the 16-bit mask never dropped a bit for the game. The change is format compliance and insurance against a large or repacked table, not a fix for an observable symptom.

What changed: the mask is 0xffffff, with the same bounds, terminator and data-range checks as before. host/oot_heart_containers.c, heart_sarc_find; the test is tests/test_heart_containers.py.

Testing

Tested: tests/test_heart_containers.py now builds a SARC with 256 KiB of filler names first, so zs.zsdic and bcett.byml.zsdic sit at word offset 0x10000 (both byte orders). On main the new cases fail for exactly the masked bits:

FAIL line 55: heart_sarc_find(p, qn, "bcett.byml.zsdic", &a, &b) && ...
FAIL line 56: heart_sarc_find(p, qn, "zs.zsdic", &a, &b) && ...

With 0xffffff: PASS: the pieces of heart in all add up ... either byte order, cut short or not; run_all --strict test_heart_containers passes.

In the game: Arch Linux, Hyprland, RTX 4070 Ti (NVIDIA 620.70.06), built as a pack over 8aa74983 (since rebased onto 1101b612, whose commits touch input, the renderer and tests, not host/oot_heart_containers.c, so the run still holds) and played — Kokiri Forest (child), 20 s measured after warm-up: 1821 frames, 91.1 fps, median 11.04 ms, 0 cold frames (base 8aa74983: 1832 frames, 91.6 fps, 10.91 ms). One run a side; the small difference is not attributable.

Still to test:

  • Windows (llvm-mingw): build and a start, expecting the same behaviour (by hand).

Measurements

Nothing to measure: the lookup the fix changes is never reached by the shipped pack (its name table is 56 bytes, far below 256 KiB); the in-game smoke above is a boot/play check and shows no attributable difference.

Risks

A malformed name offset is still refused by the same bounds checks; a valid offset past 256 KiB now resolves instead of missing (the intended change). No switch; a revert is enough. Nothing changes for saves, longshot_options.ini or the caches.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named). tests/run_all.py --strict: 811 passed, 109 failed, 18 skipped on both main @ 8aa74983 and this branch; the failing check names (108 unique) are identical on both sides (empty comm diff) — main's known-red checks and test-side drift from today's merges, none caused here.
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A fix: a game-free test that fails on main and passes here.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary `heart_sarc_find` reads a SARC `SFAT` node's name offset as `attributes & 0xffff`. In the format, bits 0-23 are the name's word offset into the `SFNT` string table and bit 24 is the has-name flag (already tested on the line above). Masking with 16 bits drops bits 16-23, so in a string table of 256 KiB or more every name past that point would resolve at the wrong offset and be looked up as absent. This is **latent, not a game bug**: the shipped `Pack/ZsDic.pack.zs` — the only file this reads — has a 56-byte name table whose largest name word offset is `0xb`, so the 16-bit mask never dropped a bit for the game. The change is format compliance and insurance against a large or repacked table, not a fix for an observable symptom. What changed: the mask is `0xffffff`, with the same bounds, terminator and data-range checks as before. `host/oot_heart_containers.c`, `heart_sarc_find`; the test is `tests/test_heart_containers.py`. ## Testing **Tested:** `tests/test_heart_containers.py` now builds a SARC with 256 KiB of filler names first, so `zs.zsdic` and `bcett.byml.zsdic` sit at word offset `0x10000` (both byte orders). On main the new cases fail for exactly the masked bits: ``` FAIL line 55: heart_sarc_find(p, qn, "bcett.byml.zsdic", &a, &b) && ... FAIL line 56: heart_sarc_find(p, qn, "zs.zsdic", &a, &b) && ... ``` With `0xffffff`: `PASS: the pieces of heart in all add up ... either byte order, cut short or not`; `run_all --strict test_heart_containers` passes. In the game: Arch Linux, Hyprland, RTX 4070 Ti (NVIDIA 620.70.06), built as a pack over `8aa74983` (since rebased onto `1101b612`, whose commits touch input, the renderer and tests, not `host/oot_heart_containers.c`, so the run still holds) and played — Kokiri Forest (child), 20 s measured after warm-up: 1821 frames, 91.1 fps, median 11.04 ms, 0 cold frames (base `8aa74983`: 1832 frames, 91.6 fps, 10.91 ms). One run a side; the small difference is not attributable. **Still to test:** - Windows (llvm-mingw): build and a start, expecting the same behaviour (by hand). ## Measurements Nothing to measure: the lookup the fix changes is never reached by the shipped pack (its name table is 56 bytes, far below 256 KiB); the in-game smoke above is a boot/play check and shows no attributable difference. ## Risks A malformed name offset is still refused by the same bounds checks; a valid offset past 256 KiB now resolves instead of missing (the intended change). No switch; a revert is enough. Nothing changes for saves, `longshot_options.ini` or the caches. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [x] Tested on my own PC, in the game for a change the game runs, as Testing says. - [x] `mise run test` passes (any failure that also fails on `main` is named). `tests/run_all.py --strict`: 811 passed, 109 failed, 18 skipped on both `main @ 8aa74983` and this branch; the failing check names (108 unique) are identical on both sides (empty comm diff) — main's known-red checks and test-side drift from today's merges, none caused here. - [x] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A fix: a game-free test that fails on `main` and passes here. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
fix(host): a SARC name offset is 24 bits, not 16
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
933a1a3aac
heart_sarc_find masked an SFAT node's name offset with 0xffff, dropping
bits 16 to 23. The format's offset is 24 bits (bits 0 to 23; bit 24 is
the has-name flag the line above already tests), so in a string table of
256 KiB or more every name past that point would resolve at the wrong
offset and be looked up as absent.

This is latent, not a game bug: the shipped Pack/ZsDic.pack.zs, the only
file this reads, has a 56-byte name table whose largest name word offset
is 0xb, so the mask never dropped a bit for the game. The fix keeps the
same bounds and terminator checks and masks the offset as the format has
it.

How to see it on main: tests/test_heart_containers.py now makes a SARC
whose wanted names sit past 256 KiB of filler names. On main both lookups
fail; with the 24-bit mask both are found in either byte order.
Author
Contributor

Tested in game: Kokiri Forest (child) under a pack over 8aa74983: 20 s after warm-up, 91.1 fps median 11.04 ms, 0 cold frames; game boots and plays. Arch Linux/Hyprland, RTX 4070 Ti, NVIDIA 620.70.06. Latent format fix; no observable change expected.

Tested in game: Kokiri Forest (child) under a pack over 8aa74983: 20 s after warm-up, 91.1 fps median 11.04 ms, 0 cold frames; game boots and plays. Arch Linux/Hyprland, RTX 4070 Ti, NVIDIA 620.70.06. Latent format fix; no observable change expected.
dazemc force-pushed fix/sarc-name-offset-24bit from 933a1a3aac
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to cf6fb40296
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 6s
2026-11-21 22:32:03 +00:00
Compare
Author
Contributor

Rebased onto 1101b612. The commits since the run (input, renderer, tests) do not touch host/oot_heart_containers.c, so the in-game smoke still holds; head is cf6fb402.

Rebased onto 1101b612. The commits since the run (input, renderer, tests) do not touch host/oot_heart_containers.c, so the in-game smoke still holds; head is cf6fb402.
Owner
@review
BeanBot left a comment

Recommendation: Approve. The code is ready: only testing is left. The synthetic test demonstrates the 24-bit offset fix, but the stated Windows build and startup check is still outstanding. The widened lookup can also accept a malformed name offset into file data. Recommend approval; not approved on the forge.

The change lets the Discord heart-count loader find dictionary names beyond 256 KiB in a SARC name table. The shipped pack does not need the fix; the added test covers the previously dropped offset bit in both byte orders.

@fix will do (1)

  1. L1 (minor) host/oot_heart_containers.c:46: The widened offset can point past the SFNT name table into the archive's file data, because it is checked against the whole buffer rather than the data start. (more on the line)

Before merging: testing (1) (runs on the systems it can change; it doesn't hold the recommendation)

  • L2 (minor): The host change has not yet been built and started on Windows with llvm-mingw, as the pull request's Still to test section acknowledges. Complete that build and startup check before merge.

Checks: Still running: CI / Conventional Commit title (pull_request), CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request), CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request), CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request), CI / Workflow lint (actionlint) (pull_request), CI / Website build and checks (pull_request), Guide screenshots / Guide images and local UI source review (pull_request), Third-party parts / As their notes say (pull_request), Third-party parts / The same as upstream (pull_request), Website / Website on Cloudflare (pull_request).

More: does it help, other systems, switches, what was read

Does it help: Yes for the synthetic case: the author reports both lookups fail on the base and pass with the fix in both byte orders. There is no claimed improvement for the shipped pack, whose reported name table is 56 bytes.

Other systems: The lookup runs while loading the Discord heart-count data, on Windows and Linux, independently of GPU vendor. The author reports a Linux game smoke test; Windows llvm-mingw build and startup remain untested. Saves, rendering and input are not changed by the diff.

Switches: None.

Reviewed: cf6fb40296 against 1101b6127e: Read AGENTS.md, CONTRIBUTING.md, the pull request template, docs/dev/standards.md, both changed files and the heart-containers header in full; read the commits, diff and relevant call-site references. Rechecked both changed files' diffs. docs/review-checklist.md and ../Tooling/AGENTS.md were absent. Other caller files were searched, not read in full; no recorded runs or generated files were included in the diff.; nothing run (a code review).

Checked and fine:

  • The synthetic names place both requested entries beyond word offset 0xffff, and the assertions check returned payloads in both byte orders.
  • The changed arithmetic is bounded on the supported 64-bit platforms; no new shared state, allocation or switch is introduced.
  • The diff contains no generated files, third-party changes or game-derived test data.

Testing: No code or tests were run during review. The author reports the new test fails on the base and passes on the change, plus a Linux game smoke test; CI remains in progress.

Overlaps and merge order: Not checked; the network is closed.


Automated code review by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.

**Recommendation: Approve.** The code is ready: only testing is left. The synthetic test demonstrates the 24-bit offset fix, but the stated Windows build and startup check is still outstanding. The widened lookup can also accept a malformed name offset into file data. Recommend approval; not approved on the forge. The change lets the Discord heart-count loader find dictionary names beyond 256 KiB in a SARC name table. The shipped pack does not need the fix; the added test covers the previously dropped offset bit in both byte orders. **`@fix` will do (1)** 1. **L1** (minor) `host/oot_heart_containers.c:46`: The widened offset can point past the SFNT name table into the archive's file data, because it is checked against the whole buffer rather than the data start. (more on the line) **Before merging: testing (1)** (runs on the systems it can change; it doesn't hold the recommendation) - **L2** (minor): The host change has not yet been built and started on Windows with llvm-mingw, as the pull request's Still to test section acknowledges. Complete that build and startup check before merge. **Checks:** Still running: `CI / Conventional Commit title (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request)`, `CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request)`, `CI / Workflow lint (actionlint) (pull_request)`, `CI / Website build and checks (pull_request)`, `Guide screenshots / Guide images and local UI source review (pull_request)`, `Third-party parts / As their notes say (pull_request)`, `Third-party parts / The same as upstream (pull_request)`, `Website / Website on Cloudflare (pull_request)`. <details> <summary>More: does it help, other systems, switches, what was read</summary> **Does it help:** Yes for the synthetic case: the author reports both lookups fail on the base and pass with the fix in both byte orders. There is no claimed improvement for the shipped pack, whose reported name table is 56 bytes. **Other systems:** The lookup runs while loading the Discord heart-count data, on Windows and Linux, independently of GPU vendor. The author reports a Linux game smoke test; Windows llvm-mingw build and startup remain untested. Saves, rendering and input are not changed by the diff. **Switches:** None. **Reviewed:** cf6fb402968b against 1101b6127e31: Read AGENTS.md, CONTRIBUTING.md, the pull request template, docs/dev/standards.md, both changed files and the heart-containers header in full; read the commits, diff and relevant call-site references. Rechecked both changed files' diffs. docs/review-checklist.md and ../Tooling/AGENTS.md were absent. Other caller files were searched, not read in full; no recorded runs or generated files were included in the diff.; nothing run (a code review). **Checked and fine:** - The synthetic names place both requested entries beyond word offset 0xffff, and the assertions check returned payloads in both byte orders. - The changed arithmetic is bounded on the supported 64-bit platforms; no new shared state, allocation or switch is introduced. - The diff contains no generated files, third-party changes or game-derived test data. **Testing:** No code or tests were run during review. The author reports the new test fails on the base and passes on the change, plus a Linux game smoke test; CI remains in progress. **Overlaps and merge order:** Not checked; the network is closed. </details> --- <sub>Automated code review by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.</sub>
@ -44,3 +44,3 @@
const uint32_t attr = rd(d, n, nodes + 16 * i + 4, 4, be);
if (!(attr & 0x01000000)) continue;
const size_t at = names + 4 * (size_t)(attr & 0xffff);
const size_t at = names + 4 * (size_t)(attr & 0xffffff);
Member

L1 (minor): The widened offset can point past the SFNT name table into the archive's file data, because it is checked against the whole buffer rather than the data start. A malformed node whose high offset lands on a matching string in file data can now be accepted as that name and return an unrelated payload. Check that the complete name and terminator lie before the SARC data offset as well as within the buffer.

**L1** (minor): The widened offset can point past the SFNT name table into the archive's file data, because it is checked against the whole buffer rather than the data start. A malformed node whose high offset lands on a matching string in file data can now be accepted as that name and return an unrelated payload. Check that the complete name and terminator lie before the SARC data offset as well as within the buffer.
HumanBean deleted branch fix/sarc-name-offset-24bit 2026-11-21 22:44:58 +00:00
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 6s
Some workflows are waiting to be reviewed.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2145
No description provided.