feat(saves): a text form of the game's saves to share without the game's files, with to-text, from-text and check-text #1403
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1403
Loading…
Reference in a new issue
No description provided.
Delete branch "mooki/Longshot:save-text-format"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: low · Tested on: Linux, Python 3.14, game-free checks on made-up saves, copies of a real finished save and six library saves, and a game run (Linux, RTX 4090) of a save made by
from-textbeside the same save as the game wrote it · Needs: WindowsWhy
A tester or a player who wants to hand someone their progress has to send the game's own save files, and the only safe way to publish one is the scrubber, which keeps a whole
progress.sav, one slot and a grey picture, for the repository only. Moving a save between Longshot, Hookshot, ReLink and NX2navi means knowing four folder layouts, and NX2navi's players already exchange a zip with the game's pictures in it. There is no way to say "this slot, at this point of the game, for this game version" in a file that is readable, has no picture, no path, no user id and no file time, and can be checked before anything is written.What changed
Builds on #1401 (the offline save editor): this PR's base is that PR's branch,
save-file-editor(head6c45402bf, onmain544229a26), forscripts/oot_savefile.py, the reader and the byte-exact writer the codec stands on. Review only the six files below; merge #1401 first, then retarget this PR tomain.A save becomes one JSON file (
*.ootsave.json, formatoot-save-textversion 1): every flag of each slot by the hash of its name and its typed value, the key run of the placed objects, the plain fields (place, hearts, rupees, the world's clock), the game version and who made it.progress.sav,account.savandoption.savcome back from it byte for byte;caption.savcomes back identical but for the picture, which is the scrubber's grey one. No game file is read.scripts/oot_save_text.py(new): the codec, onoot_savefile.SaveFile(header, table bounds, key run,murmur3,write_atomic): the table's 32 groups, 16 layouts of the data section, strings with the old bytes the game leaves after their end, values it cannot read kept as{"raw": hex}, the plain fields, validation, the canonical JSON writer.scripts/longshot_saves.py:to-text,from-text(merges by slot,--replace-all,--keep-settings,--dry-run, the guards and the backup every copy has) andcheck-text; ReLink as a program (import relink,export relink,to-text relink: the same files and user id as Longshot, in its own folder);to-text --from-zip/from-text --to-zipfor NX2navi's share zip, checked as NX2navi's import checks it.guards()andbackup_folder()arerun()'s own checks moved out;copy_savesgainsreplace_allandlabel; nothing else of the old commands changes.docs/save-text.md(new): the format, what is left out (against whatscrubremoves), the round-trip rules, the refusals; one-line pointers fromdocs/saves.mdanddocs/saves-script.md, which also gets a ReLink section.tests/test_save_text.py(new): 69 checks, with the made-up saves packed byte by byte in the test.scrub,check-repo-saves, the repository's saves rule, the game, the launcher, the website. A text file is still a save and does not make one fit for the repository.Measurements
Verdict: nothing to measure: a command line tool that never runs with the game and is in no build; no speed claim is made.
Flags and settings
One environment variable shows in the diff against
mainbecause it comes from #1401, which this PR is stacked on (documented there indocs/development.md); this PR's own six files add no switch, menu row or ini key. Its row, as #1401 has it:LONGSHOT_GAMEromfs), or the RomFS folder, thatscripts/longshot_save_editor.pyreads when--gameis not givenromfs, else asks for--gameTesting
Tested:
python -I -B tests/test_save_text.py(69 checks) andtests/test_longshot_saves.py(63),tests/test_doc_anchors.py,tests/test_longshot_save_editor.py(64, #1401's, still passing on the rebased branch),tests/run_all.py save_text longshot_saves doc_anchors, on Linux with Python 3.14. On copies, nothing posted: the finished 100% save (six slots, two cleared) and six library saves (saved by NX2navi, Hookshot and Longshot):to-textthenfrom-textgives everyprogress.sav,account.savandoption.savbyte for byte and everycaption.savequal up to its picture, with no value kept raw. Against NX2navi's ownlauncher/saves.py: the zip itsexport_savewrites is read, the zip this tool writes passes itsread_zipandimport_zip. The whole suite,tests/run_all.py --jobs 6 --strictthrough the idle gate in a fresh worktree and home: 662 passed, 44 failed, 17 skipped on this head (f57e97b2a) against 661 / 44 / 17 on its base (6c45402bf, #1401) and 660 / 44 / 17 onmain(544229a26): the same 44 fail on all three, so none is caused by this change, and the one more pass istest_save_text.py. The 44 onmaintoo, in a temporary worktree (Tooling is not beside it, fixtures that compile need build state a fresh worktree lacks, one needs this PC's CPU build): test_bug_report.py, test_bug_report_defaults.py, test_cpu_report.py, test_crash_groups.py, test_crash_record.py, test_crt_options.py, test_damage_debug_menu.py, test_display_startup_failure.py, test_dlss_feature_size.py, test_failed_compile_keeps_objects.py, test_framegen_preset.py, test_fsr_uniform_ownership.py, test_guest_imported.py, test_hardware_report.py, test_hud_layout_row.py, test_import_reach.py, test_invalidate_vcore.py, test_linux_preflight.py, test_linux_pressure.py, test_linux_update.py, test_mirror_budget.py, test_missing_nup.py, test_mods_size_table.py, test_nonuniform_image_arrays.py, test_parallel_init_order.py, test_pending_guest_import.py, test_pipeline_play_cache.py, test_present_thread.py, test_profiling_mode.py, test_readback_timeout_source.py, test_replay_write_order.py, test_rumble_menu.py, test_run_all_changed.py, test_slow_frames.py, test_staging_burst.py, test_switch_table.py, test_target_image_pool.py, test_tested_configurations.py, test_upscaler_round_trip.py, test_upscaler_rows.py, test_vcore_escape.py, test_vk_occlusion_queries.py, test_vk_swapchain_compat.py, test_walk_key.py.Game run: once, Linux, RTX 4090, windowless, build
63ddacdc9(a warm reflink clone of a built game), 17 Nov 06:15, 73 s,reports/SaveFormat-files/hold1. The library saveadult-kakariko(not a player's own: a crash-report file kept for debugging) went throughto-textthenfrom-textinto a fresh user folder with its slot_00 only (A); its own slot_00 went into another (B, the control). Each game booted from the title, pressed A on Continue and the Load card, and was in control onKakarikoVillage_Adultafter 25 s (A) and 24 s (B). The game's state read matches the source's values on both, field by field: adult, 24 of 24 quarter hearts, 77 rupees, position (1.69, 108.91, 37.06). The game's own flag query for 128 flags of the source'sprogress.sav(78 Bool, 50 Int) returned all 128 equal on A and on B. The game did not write the slot's files (hashes equal to what was put in); it added onlyaccount.sav(320 bytes), on both. No fatal, crash, device-lost or kernel driver line on either, journal complete. No speed claim is made.The game reads A's
caption.savas a 3,345-byte JPEG (the scrubber's grey one) and B's as 33,449 bytes, the same map in both. Normalized picture difference A against B: 0.001 to 0.002 on the pause pages, 0.008 in play, 0.029 30 s later (the world moves).Not tested: Windows (the folder discovery tests are Linux-only, as the script's); a game version other than the one the tool knows (2982123, game 1.1.1); the game on a text-made save of a child, of a save with an
option.savor an account whose cleared-once mark is set, or of several slots at once (this run has one adult slot and nooption.sav); a save the game has written over a text-made one, taken back to text; the pause menu's Collection and Equipment pages (the run's R presses stayed on System).Must be tested:
python -I scripts/longshot_saves.py to-text --out a.ootsave.json, thenfrom-text a.ootsave.json --to <an empty folder>, then start the game with--useron a folder holding that save, load the slot, expecting it to load at the same place with a plain grey Load-screen picture (by hand: look at the Load screen and at where Link stands; the Test Farm only starts from repository saves). The Linux RTX 4090 run is the game run above.python -I scripts/longshot_saves.py to-text --longshot <its data folder> --out a.ootsave.json, thencheck-text a.ootsave.jsonandfrom-text a.ootsave.json --to <a new folder>, expectingcheck-textto pass and everyprogress.sav,account.savandoption.savin the new folder to equal the original (fc /b) (by hand;tests/test_save_text.pyis skipped there byRUN_ALL_SKIP, astest_longshot_saves.pyis).Risks
What could break: only the new commands;
run()now callsguards()andbackup_folder(), which hold the code it had, andcopy_saveshas two new parameters with the old behaviour as default (the 63 existing checks pass unchanged).from-textreplaces slots of a real save: it backs up the whole target first, builds everything in a temp folder before touching it and swaps in by renames with rollback.Turning it off: revert the commits; nothing else depends on the new files.
Compatibility: Nothing changes for a player who does not use the new commands.
listnow also shows ReLink.Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main. (Stacked on #1401: this branch is onsave-file-editor@6c45402bf, which is onmain@544229a26; both still merge cleanly withmain@67efc4d3e, as the forge shows. This PR's own diff against #1401 is the six files above. Retarget tomainwhen #1401 lands.)python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (Named under Tested: the same 44 tests fail onmain@544229a26, on #1401's head and on this branch; none is branch-only.)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.LONGSHOT_GAMEis #1401's, documented there indocs/development.md; this PR adds no switch.)tests/that reads changed source updated in this PR, not weakened. (tests/test_save_text.pyis new and readsscripts/longshot_saves.pyandscripts/oot_save_text.py;tests/test_longshot_saves.pyis untouched and passes.)New on this forge; not opened on GitHub.
--json prints one JSON object on stdout and nothing on stderr: for set, clear, give, start and copy what changed, the files written, the backup folder and a read-back verdict; for items and places the lists; for report a schema version. A refusal is {"ok": false, "error": {"code", "message"}} with a code a program can switch on (running, other_version, out_of_range, would_overwrite ...). The text output is unchanged. docs/save-editor.md has the keys and the codes. Assisted-by: Claude Sonnet 5.5 Co-Authored-By: Claude Sonnet 5.5 <[email protected]>WIP: feat(saves): a text form of the game's saves to share without the game's files, with to-text, from-text and check-textto feat(saves): a text form of the game's saves to share without the game's files, with to-text, from-text and check-textDescription only, no code change: the description check failed because the diff against
mainincludes #1401'sLONGSHOT_GAME(this PR is stacked on it) and the Flags and settings section did not list it. It now has #1401's row for it, and the base line says both branches still merge cleanly with currentmain.Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.