WIP: fix(fs): a save reaches the save folder whole, at Commit #607
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!607
Loading…
Reference in a new issue
No description provided.
Delete branch "jarynclouatre/Longshot:fix/save-staged-commit"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: high · Tested on: Windows 11, RTX 4070 Ti SUPER; Linux x86-64 (game-free checks) · Needs: Windows and Linux, any GPU
Why
A save can be lost while it is being written. The save mount writes straight into the save folder and
Commitdoes nothing. The game saves a file by opening it, setting its size to 0, writing it in one piece and closing it; it does that forslot_NN/progress.sav,option.savandslot_NN/caption.savin turn and then callsCommit. Anything that stops the game between the resize and theCommit(a crash, a closed window, a frozen PC, a full disk under the write) leaves an empty or half-written file where the last good save was, or a newprogress.savbeside an oldcaption.sav. On the console nothing reaches the save untilCommit.The game autosaves at every area change.
Found reading
host/nn_fs.c; no issue reports a lost save. The tracker was searched for "corrupt", "save lost", "save gone", "save file".How to reproduce
Seen on: Linux x86-64, in the game-free check below (a real partial write over the real file layer). The game's order of calls is from a run on Windows 11, RTX 4070 Ti SUPER (the trace under Measurements).
Settings and switches: defaults.
Steps:
python -I -B tests/test_fs_save_commit.py --source <main's host/nn_fs.c> --case a: a 16-byte committed file, an overwrite cut off after 8 bytes by a file size limit, noCommit.Expected: the committed file still holds its old bytes.
Actual:
AssertionError: (a): failed overwrite damaged committed bytes. The game is told the write failed, and the old save is already gone.What changed
The game now writes its save into a working copy, and
Commitputs the changed files into the save folder in one step that a stop in the middle cannot leave half done. The save folder itself keeps its place and layout, so save tools, the control bridge and other programs that share the files see only whole saves.host/nn_fs.c,save_mount):user/save/<uid>stays the committed save.MountSaveDatacopies it touser/save-work/<uid>and mounts that. What was not committed is gone at the next mount, as on the console. If the working copy cannot be made, the save folder is mounted in place as before, with a log line.Commit(save_publish): for each noted path the working file is compared with the committed one; a changed file is copied beside its target as<n>.longshot-newand flushed. Then a journal (user/save-work/<uid>.journal) lists the steps, the steps are applied (renames over the targets, folders made, the game's deletions), and the journal is removed. A failure before the journal leaves the save folder untouched and returns an error; a rename the system refuses is tried again for one second, then the journal stays and the nextCommitor mount finishes it.Commitwith a file still open for writing is refused, as Horizon refuses it; the game never does that (trace below).<uid>.journal.failedwith a log line and the game goes on with the files as they are.Commitcannot publish the mixed bytes.user/save-work/<uid>.lockis held by the first. A second one mounts the save folder in place, as both do on main, and never touches the working copy or the journal.HOS_FS_TRACE=1now also logs each change to a writable mount and eachCommitwith its counts and time.scripts/longshot_saves.pydoes not takesave-workfor a save and refuses to copy a save whose journal is waiting; the Linux move of the data folder (host/hos_main.c) leavessave-workbehind and waits while a journal is there.docs/saves.mdsays what the folder is.tests/test_fs_save_commit.pyruns the real HLE functions overhost/posix/win32_posix.c.tests/test_guest_copies.pyandtests/test_vcore_fs_open.pyfollow the moved code; the second also checks that an open on the RomFS mount takes no save lock.Reviewer notes:
host/nn_fs.cgrows by about 950 lines. #61, #68, #355 and #461 change the same file and will conflict in text; #68 and #355 copy the save folder after a quiet time because writes were in place, and with this change the save folder only ever holds whole saves. I will rebase onto whichever lands first.Measurements
Verdict: the new check fails on main and passes here, and the suite has no new failures. In the game 12 saves in a row were published whole, each in 12.8 to 15.4 ms (median 13.7 ms) on the game's own save thread. Frame times around a save were not compared with main yet, which is why this is WIP.
Hardware: Windows 11, RTX 4070 Ti SUPER, Ryzen 7 9800X3D, 32 GB, NVMe drive; the suite on a Linux x86-64 PC (gcc 15.2, Python 3.14.4)
Scene: a scripted walk of about 9 minutes with debug teleports (Graveyard, Kakariko, Hyrule Field, Gerudo Valley), headless, after a shader preparation
Method: 2026-11-14. Game: one run with
HOS_FS_TRACE=1, thefs:lines read from the log, the two folders listed after the run. That run was made before the rebase, on main @8b619764with this change;host/nn_fs.candhost/posix/win32_posix.care the same in this commit. Suite:python3 -I -B tests/run_all.py --jobs 4 --timeout 120, main and the PR in turn, two runs each.How to reproduce:
HOS_FS_TRACE=1inlongshot_env.txt, walk through a few loading zones, read thefs: Commitandfs: save:lines225c37f8e7e7e368tests/run_all.py(run 1, run 2)test_appimage_player_build.py,test_bake_portability.py,test_cpm_discovery.py,test_crash_dialog.py,test_env_on.py,test_lift_cleanup_error.py,test_make_game_cache.py,test_readback_late.py,test_vk_pgo.py,test_vram_budget_row.py,test_walk_key.pytest_fs_save_commit.py --case atest_fs_save_commit.py, all casesIn the game (this change only; main logs none of these lines):
8b619764account.savwith the same bytes at the start (nothing to publish)progress.sav(about 124 KB),option.sav(42 KB),caption.sav(about 38 KB); thenCommitgmd::SaveDataMgrfor every write and everyCommitCommitwith a file open for writingsaveandsave-workhold the same files and sizes; no journal, no staged fileFlags and settings
No new switch. One existing switch logs more.
HOS_FS_TRACECommitTesting
Tested: the game-free suite on Linux, as in Measurements.
tests/test_fs_save_commit.py: a failed partial write, three files published together, a remount withoutCommit, a stop after the journal and a stop in the middle of applying it (both replayed at the next mount), half-written and hostile journals, deletes and renames, a file the player put into the save folder during play, a publish that runs out of room, a working copy that cannot be made, a refused rename, two real processes on one save, links below both folders, and that the build finds every changed function asHLE(name). The game on Windows 11 with an RTX 4070 Ti SUPER: the run under Measurements. The Windows game of that run was built from this code with llvm-mingw 20261104 (before the rebase). For Linux,host/nn_fs.candhost/posix/win32_posix.ccompile whole with gcc 15.2 (-Wall -Wextra) in the new check, and the Linux move inhost/hos_main.cintests/test_linux_saves_move.py; no full Linux game was built.Not tested: frame times around a save against main. A real power cut or reset in the middle of a save (the checks stop the process at those points instead). The game on Linux. A save folder on a network drive or under a cloud sync program. A second copy of the game on Windows (checked on Linux with two processes; on Windows read in the source only).
Must be tested:
fs: save: publishedlines withHOS_FS_TRACE=1; I will post a main-against-PR run)fs: save: published 3 fileslines in the log (by hand)Risks
What could break: saving, for everyone. A fault here could keep saves from reaching the save folder or keep the game from starting with one. The ways out built in: any failure to set up the working copy mounts the save folder in place as before; a journal that cannot be replayed is moved aside and play goes on; a
Committhat fails returns an error to the game and leaves the last good save alone. The publish adds about 14 ms on the game's save thread per save on the test PC.Turning it off: revert the commit. There is no switch; a save folder written by this change is an ordinary save folder, so main reads it as it is.
Compatibility: saves keep their place and format. A new folder
save-workappears besidesavein the user folder; it can be deleted while the game is closed. Progress the game wrote without aCommitis no longer kept at the next start; the game commits every save (13 of 13 in the trace). Tools that copy saves should copysaveonly.Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
mainand passing here.tests/that reads changed source updated in this PR, not weakened.e7e7e368050f30a75f15Measured on Windows 11, RTX 4070 Ti SUPER, Ryzen 7 9800X3D, no window, the game at its 60 fps. Main
225c37f8against this branche7e7e368, both with main's renderer DLL, in turn (main, PR, main, PR). The route is a scripted adult walk of about nine minutes with four teleports (Graveyard, Kakariko Village, Hyrule Field, Gerudo Valley) and 13 autosaves.HOS_PRESENT_TRACEgave the present intervals; a file watcher on the user folder gave each save's time on the same clock.Frames around a save (from 0.1 s before the first file change of a save to 0.4 s after the last):
The PR's windows are longer because they hold the staged writes and then the Commit. The 26 Commits with files took 11.4 to 14.9 ms each (13.1 ms mean), on the game's save thread.
The first 229 s of the route, where all four runs do the same thing:
After 230 s the runs are not comparable as wholes: in both PR runs Link's walk through the Graveyard goes another way than in both main runs (the PR runs get one more autosave at 254 s and load more actors), so the later counts differ with the path. The inputs are timed by the clock, and I have not yet shown whether that is the route reacting to a frame of timing or something this change does. I am leaving WIP on until I have.
A stop in the middle of a save (PR): the game was ended at the first change in
save-workof the third save. The save folder was byte for byte what it was before that save (14 files, same hashes),save-workheld the half-made save, and no staged or journal file was left. On the next start the game read the save folder's save andsave-workmatched it again.Not tested: Linux play.
Thanks for taking this on. A save lost halfway through its write is about the worst bug a player can meet; a working copy with a journal applied at
Commitis a clear way to stop it. I read the diff and testede7e7e368and its rebase onto main,0f30a75f(same changed lines,host/nn_fs.cthe same file byte for byte), on Linux. In every kill test I made on your head, a save ends whole old or whole new.tests/test_fs_save_commit.pynn_fs.cit fails at case (a), as you wrotetest_guest_copies,test_vcore_fs_open,test_linux_saves_move,test_longshot_saves,test_fs_linksrun_all.py --stricttest_vcore_fs_openonly (missing_CRT_SECURE_NO_WARNINGSin the fixture), the same as on the merge base; the other drivers skip Windows.host/nn_fs.ccompiles for Windows (syntax only)A kill at every file call. One autosave runs through your real
nn_fs.coverhost/posix/win32_posix.c. The process is stopped before each call that changes the user folder (23 calls in the mount and 38 in the autosave for an overwrite; 56 in all for a first save), then the next start runs and the three files are compared byte for byte with the old and new ones.Writes cut in half: 32 runs, all old. The next start killed at each of its own calls: 2,894 runs, 2,568 old and 326 new, none mixed or torn. No stray or zero-filled file is left. A second program with random game calls: 6,269 kills, always the tree before or after.
In the real game (built from
e7e7e368and its merge base225c37f8; killed from outside). A normal autosave goes through the working copy:fs: Commit: 3 files, 3 ops, 0 failed paths, 0 open writers, result 0(16 and 35 ms in my two complete autosaves). Killed at the first write into the working copy, at the first staged file, or whenjournal.tmpis made, the next start loads the old save. Killed when the journal is renamed in (step 52), at the first or last apply rename, or when the journal is removed, it loads the new one (all but the last log the replay at the mount). Each time the three files equal the old or new bytes and no stray is left. Both apply kills came after all three renames, so the mixed window is only in the sweep. The merge base killed in its own write keepscaption.savempty,progress.savandoption.savnew, and the next start leaves it so.Found (each reproduced against your real
nn_fs.c)host/nn_fs.c:922,:936: when the working copy cannot be made whole (disk nearly full), the half copy stays, the mount writes in place instead, and the half copy takes the room those writes need. With a 304 KB save the merge base tears at up to 32 KB free; your head tears at up to 300 KB, keeps the old save from 304 to 548, and saves from 552. One line after the copy,if (!ok) save_remove(work);, brings it back to the merge base's 32 KB, and your test still passes.:401,:922: one committed entry whose namesave_relativerefuses (a:b,x|y,what?, valid on Linux) fails the whole copy, and the session runs in place unprotected. Leave such entries out of the copy, as links are.:400,:559,:584,:784: a link insidesave/<uid>(a slot on another disk) hides that slot from the game and makes everyCommitfail, sooption.savis not published either. The merge base followed the link. Mount in place when the copy skips a link.:906-917: a replay that fails at the mount after a partial apply retires the journal and deletes the staged files, so the save stays half new for good. Keep the journal and mount in place, and a later start can finish it.:1281-1284: after a failed write whose put-back also failed, the next good rewrite of that file is undone at itsCloseFile. Restoring inOpenFilefor write would fix it.Smaller: on Linux the copies lose the source's mode (a 0600 save is 0644 after the first
Commit). Between the journal's rename and the last (steps 55 and 56) the folder holds a newcaption.savbeside an oldprogress.sav. Read, not run, on Windows: every reparse point counts as a link (OneDrive placeholders), the journal's rename and delete get no retry, the lock file opens with sharing 0. Whether the game tolerates a non-zeroCommit(0x2EE202) I could not read.Not covered, and a separate change could: the settings files. On main
longshot_options.iniis replaced whole by #550, but its new file is not flushed to the disk before the move and a kill leaves its.newbehind;language.txtandmouse_sens.txtare written in place, and #729's one-time options copy is a plainCopyFileA. I will take those in their own change, so yours stays one topic.Rebase: main (
f637b42d) has moved since0f30a75fand now conflicts inhost/hos_main.c(a line of the saves-move loop that #557 changed) andtests/test_linux_saves_move.py. Keeping both sides resolves both, and the focused tests pass that way.Not tested: Windows, apart from your one stop in a save (the forge skipped its Python checks on this head; its Guide screenshots check fails the same way on the base), a power cut, a second copy of the game on a real save.
Your test lets 16 of 28 one-line breakages of
nn_fs.cthrough, six of them removed flushes, which only a check of the flush order sees. If you want them, I will send that check and the kill test for your branch (Linux only).A Steam Deck run, as a tester's report: pass on everything we could reach. No save file is written in place, every kill during a save left the saves whole, and main's controls lose a file each time.
Hardware: Steam Deck OLED, SteamOS 3.9.5 (kernel 7.2.7-valve2), saves on the internal SSD (ext4). Headless over ssh with the control bridge, scratch copies of a save only.
Build: your head
0f30a75fcherry-picked onto an older main (9c9d69ed) that we have a warm Deck build of;host/nn_fs.c,host/posix/win32_posix.cand your test are byte-identical to your head. The save tool's part (scripts/longshot_saves.py) isn't in that build. Control: main9c9d69editself.Method:
straceof the game's file calls withHOS_FS_TRACE=1; the kills areSIGKILLinjected by strace on entry to an exact call on an exact path. After each kill the game is started again and every file is checked against the before and after hashes, and the slot list and Continue are checked through the bridge.The file calls of a save
rename5,fsync12,fdatasync8; nothing written in place insave/ftruncate(0)+pwrite64in place onprogress.sav,caption.sav,option.sav; no renameaccount.sav)Commit: 0 files ... result 0save/save-work/Without strace the two Commits of a save take 20.8 to 26.0 ms each (
fs: Commit: 2 files, 2 ops ... result 0 in 26.031 ms, then1 files ... 20.754 ms). So the cost is about 45 ms a save and under 11 ms at start-up.One thing worth knowing: the game saves in two Commits: the slot (
progress.sav,caption.sav) first, thenoption.sav, each with its own journal. A kill between them leaves the slot new andoption.savold. That is what the PR promises (each Commit whole), and the game loaded it fine, but "a save" as the player sees it is not one transaction.Kills during a save (a manual overwrite of a slot), then the next start
option.sav's writeoption.savcut to 0 in the work copy onlyoption.savoldpublished 2 files in 12.3 msbefore the mountpublished 2 files in 14.4 mspublished 2 files in 13.4 msoption.sav's rename (Commit 2)option.savstagedpublished 1 files in 12.9 mspublished 2 files in 8.9 ms(finds them done)Eleven points, eleven passes. No staged file or journal was left after any next start, and none of the PR's failure lines appeared in any run.
main, the same kills between the cut to 0 and the write:
progress.savCRASH: exception 0xc0000005) and it freezescaption.savoption.savoption.savstays emptyNot covered
result 0).rename()over the existing file, and on those file systems a rename over an existing file is not atomic on disk: a pulled card or a power cut during it can leave neither name. A kill of the game cannot cause that, only a stop of the drive. Neither the description nordocs/saves.mdmentions it; one line there would set the right expectation for SD cards.fsyncof theuser/folder or its parent was seen in the traces (the save folder, the profile folder, the slot folder andsave-workare flushed). We don't know that it matters; noting it in case you expected one.Of the five bugs in mooki's review, none can be triggered by this test (plenty of disk, only the game's own file names, no links in the save, and a kill cannot make a rename or a write fail), so this run says nothing about them either way.
Posted by an AI agent (Claude Code) for zeldaootr.
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.