MagicBeanDev/Hookshot
Archived
Watch
6
0
Fork
You've already forked Hookshot
 0
Code Issues 125 Pull requests 205 Activity

WIP: feat: add standalone save editor and read-only diagnostics #784

Closed
nicholasgower wants to merge 5 commits from feat/save-editor into main
pull from: feat/save-editor
merge into: MagicBeanDev:main
Conversation 3 Commits 5 Files changed 27 +6917 -7
nicholasgower commented 2026-11-08 21:05:18 +00:00 (Migrated from github.com)

What this adds

  • save_editor/: a stdlib-only Python editor for Hookshot's GameData .sav files.
    • Typed section parser/writer with byte-preserving scalar and fixed-string edits.
    • Atomic writes and automatic backups; refuses to edit while the game runs.
    • list, check, dump, get, set, hash, scan, names, backup.
    • Names/schema tables (JSON or TSV), auto-discovery, enum labels.
  • Read-only diagnostics:
    • compare.py: file/slot/account diff with --json.
    • schema_report.py: name/type/version coverage.
    • acceptance.py: parse/round-trip and progress/caption pair checks.
  • tests/run_all.py now discovers save_editor/tests/test_*.py; contributing and workflow text updated.
  • Tests: synthetic fixture plus an opt-in real-save round-trip suite. No game data, saves, schemas or backups are included.

Verification

  • python -I -B tests/run_all.py save_editor -> 5 checks pass.
  • Read-only real-tree check: 14 files, 0 bad; acceptance --pair passes; newest slot reported.
  • compare and schema_report exercised against real saves.

Notes

  • The editor lives outside the game build and is not wired into nx2/build.py or CMake.
  • In-game saving creates/rotates a slot; it does not overwrite a chosen save. The editor does not assume otherwise.
## What this adds - `save_editor/`: a stdlib-only Python editor for Hookshot's GameData `.sav` files. - Typed section parser/writer with byte-preserving scalar and fixed-string edits. - Atomic writes and automatic backups; refuses to edit while the game runs. - `list`, `check`, `dump`, `get`, `set`, `hash`, `scan`, `names`, `backup`. - Names/schema tables (JSON or TSV), auto-discovery, enum labels. - Read-only diagnostics: - `compare.py`: file/slot/account diff with `--json`. - `schema_report.py`: name/type/version coverage. - `acceptance.py`: parse/round-trip and progress/caption pair checks. - `tests/run_all.py` now discovers `save_editor/tests/test_*.py`; contributing and workflow text updated. - Tests: synthetic fixture plus an opt-in real-save round-trip suite. No game data, saves, schemas or backups are included. ## Verification - `python -I -B tests/run_all.py save_editor` -> 5 checks pass. - Read-only real-tree check: 14 files, 0 bad; `acceptance --pair` passes; newest slot reported. - `compare` and `schema_report` exercised against real saves. ## Notes - The editor lives outside the game build and is not wired into `nx2/build.py` or CMake. - In-game saving creates/rotates a slot; it does not overwrite a chosen save. The editor does not assume otherwise.
nicholasgower commented 2026-11-08 21:50:19 +00:00 (Migrated from github.com)

I'm actively completing the remaining steps in the roadmap in this PR.

I'm actively completing the remaining steps in the roadmap in this PR.
nicholasgower commented 2026-11-08 23:10:16 +00:00 (Migrated from github.com)

Save editor: Option A progress and current blockers

Summarising where this PR stands and what is still open.

Done (head e577e9c)

  • save_editor/import_names.py — merges the TotK CSV/WordList, the OoT binary scan and a curated overlay; every exact name is validated with murmur3_32(name) == hash; provenance is totk / oot-scan / curated / semantic; priority curated > scan > TotK. Read-only for saves and game files.
  • save_editor/label_diff.py — labels hashes that changed between two saves (--hash, --name, --force, --dry-run, --json), writing only to --out.
  • save_editor/schema_report.py — provenance counts, --by-type, --by-save-file, --named-only, and a critical-field checklist.
  • Scanned strings are filtered to plausible GameData paths, so incidental printable hash matches (e.g. ) (@) cannot become names. This removed one false exact name.
  • OoT-specific curation: 21 semantic labels for equipment, capacity upgrades, item/quest slots, chest, sequence, minigame and dungeon progress. The table and the evidence for each are in save_editor/ROADMAP.md. The overlay itself stays in %TEMP%; no generated names/schema is committed.

Current numbers (Product.110, real save tree)

  • 12,019 fields: 218 exact names (207 validated TotK + 11 OoT) and 21 semantic.
  • 14 save files / 71,888 entries; 1,369 named entries (1,243 exact + 126 semantic, repeated per slot).
  • 0 type mismatches, 0 version mismatches; all 10 critical fields OK.
  • acceptance --pair passes; the 9 save-editor checks pass.

Blockers

  1. Exact names for most hashes are not recoverable from the retail build. There is no HashText; only 11 full OoT paths are printable. A bounded path reconstruction was tried and is not usable: 16.7M candidate paths produced 46 hits, which is the ~47 expected by chance against 12,019 hashes, so those hits are collisions, not names. That is why the remaining OoT fields are semantic labels rather than exact ones.
  2. The semantic labels are not yet confirmed in game. They are evidence-based (the value is an Item_*/Weapon_* asset path, or the field's enum labels are self-describing), but they are working names, not official paths.
  3. Automating the in-game check is the sticking point. The game boots against a separate --user copy, reaches the Load screen and loads the edited slot_05 (Kakariko), but synthetic input does not reliably open the pause menu and desktop capture is racy. Confirming equipment/upgrades needs direct button injection plus framebuffer screenshots (or a save-state/reload hook); the desktop-keyboard route is not good enough.
  4. Songs / medallions / skulltula tokens / trials are known as values (Item_Melody_*, Item_Medal_*, Item_Stone_*, Item_Token_*, DungeonBoss*), but the fields that store them cannot be told apart offline. They are expected to share the item-path/item-enum slots once collected.

Next

  • Confirm the visible labels (sword/shield, ocarina, wallet cap) by editing one field at a time on a copy, reloading, and reading the Equipment/Items/Quest screens.
  • Leave fields that need a specific event (chest, sequence, minigame, boss progress) marked semantic/unconfirmed unless a prepared state is available.
  • Then refresh the coverage report and docs; the overlay remains outside the repo.

No game data, saves or generated schemas are included in this PR.

### Save editor: Option A progress and current blockers Summarising where this PR stands and what is still open. **Done (head `e577e9c`)** - `save_editor/import_names.py` — merges the TotK CSV/WordList, the OoT binary scan and a curated overlay; every exact name is validated with `murmur3_32(name) == hash`; provenance is `totk` / `oot-scan` / `curated` / `semantic`; priority curated > scan > TotK. Read-only for saves and game files. - `save_editor/label_diff.py` — labels hashes that changed between two saves (`--hash`, `--name`, `--force`, `--dry-run`, `--json`), writing only to `--out`. - `save_editor/schema_report.py` — provenance counts, `--by-type`, `--by-save-file`, `--named-only`, and a critical-field checklist. - Scanned strings are filtered to plausible GameData paths, so incidental printable hash matches (e.g. `) (@`) cannot become names. This removed one false exact name. - OoT-specific curation: **21 `semantic` labels** for equipment, capacity upgrades, item/quest slots, chest, sequence, minigame and dungeon progress. The table and the evidence for each are in `save_editor/ROADMAP.md`. The overlay itself stays in `%TEMP%`; no generated names/schema is committed. **Current numbers (Product.110, real save tree)** - 12,019 fields: **218 exact names** (207 validated TotK + 11 OoT) and **21 semantic**. - 14 save files / 71,888 entries; 1,369 named entries (1,243 exact + 126 semantic, repeated per slot). - 0 type mismatches, 0 version mismatches; all 10 critical fields `OK`. - `acceptance --pair` passes; the 9 save-editor checks pass. **Blockers** 1. **Exact names for most hashes are not recoverable from the retail build.** There is no `HashText`; only 11 full OoT paths are printable. A bounded path reconstruction was tried and is **not usable**: 16.7M candidate paths produced 46 hits, which is the ~47 expected by chance against 12,019 hashes, so those hits are collisions, not names. That is why the remaining OoT fields are semantic labels rather than exact ones. 2. **The semantic labels are not yet confirmed in game.** They are evidence-based (the value is an `Item_*`/`Weapon_*` asset path, or the field's enum labels are self-describing), but they are working names, not official paths. 3. **Automating the in-game check is the sticking point.** The game boots against a separate `--user` copy, reaches the Load screen and loads the edited `slot_05` (Kakariko), but synthetic input does not reliably open the pause menu and desktop capture is racy. Confirming equipment/upgrades needs direct button injection plus framebuffer screenshots (or a save-state/reload hook); the desktop-keyboard route is not good enough. 4. **Songs / medallions / skulltula tokens / trials** are known as *values* (`Item_Melody_*`, `Item_Medal_*`, `Item_Stone_*`, `Item_Token_*`, `DungeonBoss*`), but the fields that store them cannot be told apart offline. They are expected to share the item-path/item-enum slots once collected. **Next** - Confirm the visible labels (sword/shield, ocarina, wallet cap) by editing one field at a time on a copy, reloading, and reading the Equipment/Items/Quest screens. - Leave fields that need a specific event (chest, sequence, minigame, boss progress) marked `semantic`/unconfirmed unless a prepared state is available. - Then refresh the coverage report and docs; the overlay remains outside the repo. No game data, saves or generated schemas are included in this PR.
NotRexed commented 2026-11-09 05:32:55 +00:00 (Migrated from github.com)

Thank you! The save editor's tests pass on current main and it stays out of the game build, but the PR is still marked as a draft (you said you're finishing the remaining steps). Closing to keep the queue clear; please reopen it, out of draft, when it's ready, and it can be merged.

Thank you! The save editor's tests pass on current main and it stays out of the game build, but the PR is still marked as a draft (you said you're finishing the remaining steps). Closing to keep the queue clear; please reopen it, out of draft, when it's ready, and it can be merged.
Commenting is not possible because the repository is archived.
Reviewers
No reviewers
Labels
Clear labels   
accessibility
Barrier affecting people with disabilities

  
autorelease: pending

   
autorelease: tagged

   
bug
Something isn't working

  
dependencies
Pull requests that update a dependency file

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
github_actions
Pull requests that update GitHub Actions code

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
testing wanted

   
wontfix
This will not be worked on

No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Hookshot!784
No description provided.