WIP: fix(host): closing the game waits for a save it is writing #1422

Draft
TrashPanda wants to merge 2 commits from TrashPanda/Longshot:fix/close-waits-for-save into main
Contributor

Type: fix · Risk: low · Tested on: Linux, game-free only · Needs: any PC, closing the game while it saves, in the game

Why

nn::fs::Commit does nothing here, so a save goes straight into its files. Every close (the window, the control bridge's quit, the Windows stop worker) runs hos_close_finish and ends the process without waiting for a file the game is writing; the log even says stop: close work done (the game's saves are not waited for). Closing the window during a save could cut the save file short. Hookshot's issue 32 was that report.

Ported from Hookshot 7d22d4fb (pixl27): git.notrexed.dev/Hookshot/Hookshot@7d22d4fb9d

The tracker was searched for "saves are not waited", "save in progress" and hos_close_finish: no pull request or issue is this fix. The issues that match only quote the close line in their logs; #102 added the windowless stop this now waits in.

How to reproduce

Seen on: Hookshot's report. Not run on Longshot.
Settings and switches: defaults.
Steps: save in the game and close the window while the save is being written.
Expected: the save is whole on the next start.
Actual: on main the process exits at once, whatever the game is writing.

What changed

host/nn_fs.c counts the files open for writing and when one was last written (WriteFile, before and after; SetFileSize). hos_fs_wait_writes waits, 10 ms at a time and at most 3 s, while a file is open for writing and was written in the last second, and logs stop: the game was writing a file: waited N ms for it when it waited. hos_close_finish calls it first, so all three closes wait. The game's threads go on meanwhile.

  • host: host/nn_fs.c, host/hos_relaunch.c (the close line now reads stop: close work done (a file the game was writing waited for, at most 3 s)).
  • test: tests/test_close_waits_for_save.py (new) compiles the wait with a made-up clock and reads the four fs calls. tests/test_windows_stop.py, tests/crash_record_test.c and tests/test_vcore_fs_open.py get a stand-in for the new names; tests/windows_stop_test.c quotes the new close line.
  • docs: docs/development.md names the new line and counts the wait against an OS-ending event's four seconds.

Measurements

Verdict: the game-free check fails before the fix and passes here. Nothing in the game has been measured.

Hardware: Linux, no game run.
Scene: none.
Method: python -I -B tests/test_close_waits_for_save.py on the test commit and on this head.
How to reproduce: check out 11345ca0 (the test alone, on main 544229a2) and run the command; then this head.

Before: main @ 544229a2 plus the test @ 11345ca0 After: this PR @ 26daec6e
tests/test_close_waits_for_save.py fails: closing does not wait for a file the game is writing PASS

python -I -B tests/run_all.py on this head (Linux, 16 at once): 603 passed, 38 failed, 81 skipped (no Vulkan or SPIR-V headers here). The failures are the ones main 544229a2 has on this PC, except tests/test_perf_report.py, which failed under the full run's load and passes alone on this head and on main.

Flags and settings

None: no switch. The wait is at most 3 s, and only while the game wrote a file in the last second.

Testing

Tested: Linux, game-free. The new check, and the checks that compile hos_close_finish, OpenFile or read nn_fs.c (test_windows_stop, test_crash_record, test_vcore_fs_open, test_fs_links, test_fs_path_utf8, test_guest_copies, test_longshot_saves, test_options_path, test_menu_restart, test_quit_not_crash, test_longshot_ctl).

Not tested: the game, Windows, a host build (a64_suyu.h is not on this PC). Opened with --force because Joey asked; this PC is not running the game.

Must be tested:

  • Windows and Linux: save, then close the window at once, several times. Expected: the save loads on the next start, and the log has stop: the game was writing a file: waited N ms for it when the close came during a write. A close with nothing being written shows no wait. Test Farm: @Beanstalk save and close the window right after, Windows and Linux, expecting the save to load next start.

Risks

What could break: a close can take up to 3 s longer when a file was written in the last second. A game that keeps a file open for writing and writes it all the time would always wait the full 3 s. On a Windows logoff or shutdown the wait comes out of the four-second budget after rx_close_asked, so a slow pipeline-cache save plus a game write could reach the deadline (exit 124), which today would also cut the write.

Turning it off: none as a switch; reverting the commit restores the immediate exit.

Compatibility: nothing in saves or options. MKWVK_API_VERSION is unchanged.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why. Not ticked yet: Testing's Tested is game-free only and says the game was not run, though host/ is a game file.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks. Not ticked yet: Risks says a close can take up to 3 s longer, and Measurements shows no close time measured.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). The 38 failures fail on main here too; test_perf_report failed only under load.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. Not built here; the new code compiles in the check's cut with -Wall -Wextra -Werror, and it uses only __atomic builtins, which both compilers have.
**Type:** fix · **Risk:** low · **Tested on:** Linux, game-free only · **Needs:** any PC, closing the game while it saves, in the game ## Why `nn::fs::Commit` does nothing here, so a save goes straight into its files. Every close (the window, the control bridge's quit, the Windows stop worker) runs `hos_close_finish` and ends the process without waiting for a file the game is writing; the log even says `stop: close work done (the game's saves are not waited for)`. Closing the window during a save could cut the save file short. Hookshot's issue 32 was that report. Ported from Hookshot `7d22d4fb` (pixl27): https://git.notrexed.dev/Hookshot/Hookshot/commit/7d22d4fb9d754b09cdf66d1644a1f2044b36e073 The tracker was searched for "saves are not waited", "save in progress" and `hos_close_finish`: no pull request or issue is this fix. The issues that match only quote the close line in their logs; #102 added the windowless stop this now waits in. ## How to reproduce **Seen on:** Hookshot's report. Not run on Longshot. **Settings and switches:** defaults. **Steps:** save in the game and close the window while the save is being written. **Expected:** the save is whole on the next start. **Actual:** on main the process exits at once, whatever the game is writing. ## What changed `host/nn_fs.c` counts the files open for writing and when one was last written (`WriteFile`, before and after; `SetFileSize`). `hos_fs_wait_writes` waits, 10 ms at a time and at most 3 s, while a file is open for writing and was written in the last second, and logs `stop: the game was writing a file: waited N ms for it` when it waited. `hos_close_finish` calls it first, so all three closes wait. The game's threads go on meanwhile. - host: `host/nn_fs.c`, `host/hos_relaunch.c` (the close line now reads `stop: close work done (a file the game was writing waited for, at most 3 s)`). - test: `tests/test_close_waits_for_save.py` (new) compiles the wait with a made-up clock and reads the four fs calls. `tests/test_windows_stop.py`, `tests/crash_record_test.c` and `tests/test_vcore_fs_open.py` get a stand-in for the new names; `tests/windows_stop_test.c` quotes the new close line. - docs: `docs/development.md` names the new line and counts the wait against an OS-ending event's four seconds. ## Measurements **Verdict:** the game-free check fails before the fix and passes here. Nothing in the game has been measured. Hardware: Linux, no game run. Scene: none. Method: `python -I -B tests/test_close_waits_for_save.py` on the test commit and on this head. How to reproduce: check out `11345ca0` (the test alone, on main `544229a2`) and run the command; then this head. | | Before: main @ 544229a2 plus the test @ 11345ca0 | After: this PR @ 26daec6e | |---|---|---| | `tests/test_close_waits_for_save.py` | fails: closing does not wait for a file the game is writing | PASS | `python -I -B tests/run_all.py` on this head (Linux, 16 at once): 603 passed, 38 failed, 81 skipped (no Vulkan or SPIR-V headers here). The failures are the ones `main` `544229a2` has on this PC, except `tests/test_perf_report.py`, which failed under the full run's load and passes alone on this head and on `main`. ### Flags and settings None: no switch. The wait is at most 3 s, and only while the game wrote a file in the last second. ## Testing **Tested:** Linux, game-free. The new check, and the checks that compile `hos_close_finish`, `OpenFile` or read `nn_fs.c` (`test_windows_stop`, `test_crash_record`, `test_vcore_fs_open`, `test_fs_links`, `test_fs_path_utf8`, `test_guest_copies`, `test_longshot_saves`, `test_options_path`, `test_menu_restart`, `test_quit_not_crash`, `test_longshot_ctl`). **Not tested:** the game, Windows, a host build (`a64_suyu.h` is not on this PC). Opened with `--force` because Joey asked; this PC is not running the game. **Must be tested:** - Windows and Linux: save, then close the window at once, several times. Expected: the save loads on the next start, and the log has `stop: the game was writing a file: waited N ms for it` when the close came during a write. A close with nothing being written shows no wait. Test Farm: @Beanstalk save and close the window right after, Windows and Linux, expecting the save to load next start. ## Risks **What could break:** a close can take up to 3 s longer when a file was written in the last second. A game that keeps a file open for writing and writes it all the time would always wait the full 3 s. On a Windows logoff or shutdown the wait comes out of the four-second budget after `rx_close_asked`, so a slow pipeline-cache save plus a game write could reach the deadline (exit 124), which today would also cut the write. **Turning it off:** none as a switch; reverting the commit restores the immediate exit. **Compatibility:** nothing in saves or options. `MKWVK_API_VERSION` is unchanged. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [ ] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. Not ticked yet: Testing's Tested is game-free only and says the game was not run, though host/ is a game file. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [ ] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. Not ticked yet: Risks says a close can take up to 3 s longer, and Measurements shows no close time measured. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). The 38 failures fail on `main` here too; `test_perf_report` failed only under load. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. - [ ] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. Not built here; the new code compiles in the check's cut with `-Wall -Wextra -Werror`, and it uses only `__atomic` builtins, which both compilers have.
nn::fs::Commit does nothing here, so a save goes straight into its files,
and every close (the window, the control quit, the Windows stop worker)
ends the process without waiting for one in progress. This check fails
until hos_close_finish waits, up to 3 s, while a file open for writing was
written in the last second, and compiles that wait with a made-up clock.

Ported-from: 7d22d4fb9d
Co-authored-by: pixl27 <[email protected]>
Assisted-by: Claude Opus 5.5
fix(host): closing the game waits for a save it is writing
Some checks are pending
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 8s
26daec6e82
nn::fs::Commit does nothing here, so a save goes straight into its files,
and closing the window, the control bridge's quit or a Windows stop could
end the process in the middle of one. nn_fs.c now counts files open for
writing and when one was last written; hos_close_finish, the tail every
close runs, waits up to 3 s while such a file was written in the last
second, and logs how long. The close log line says so instead of "the
game's saves are not waited for", and docs/development.md counts the wait
against an OS-ending event's four seconds. The checks that compile
hos_close_finish and OpenFile get a stand-in for the new names.

Ported-from: 7d22d4fb9d
Co-authored-by: pixl27 <[email protected]>
Assisted-by: Claude Opus 5.5
Author
Contributor

@Beanstalk Windows and Linux, any GPU: from a new save, save in the game, then close the window right after the save starts, several runs; start again each time. Expected: the save loads on the next start, and when the close came during a write the log has stop: the game was writing a file: waited N ms for it (at most 3000 ms). A close with nothing being written shows no wait and closes as fast as main. Not run in the game yet; Joey asked for this pull request so the farm can take that run.

@Beanstalk Windows and Linux, any GPU: from a new save, save in the game, then close the window right after the save starts, several runs; start again each time. Expected: the save loads on the next start, and when the close came during a write the log has `stop: the game was writing a file: waited N ms for it` (at most 3000 ms). A close with nothing being written shows no wait and closes as fast as main. Not run in the game yet; Joey asked for this pull request so the farm can take that run.
Member

@TrashPanda only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@TrashPanda only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Collaborator

Possibly related, for context:

  • PR #607 WIP: fix(fs): a save reaches the save folder whole, at Commit — Overlapping save-file handling in host/nn_fs.c; likely merge conflict.
Possibly related, for context: - PR #607 WIP: fix(fs): a save reaches the save folder whole, at Commit — Overlapping save-file handling in host/nn_fs.c; likely merge conflict.
Owner

Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).

  • Tested on my own PC: not ticked: Testing's Tested is game-free only and says the game was not run, though host/ is a game file. Run it yourself on your PC (in the game, for a change the game runs) and say where and what you saw in Testing's Tested; or, when the fix's own case can't be run by hand, add a **Not testable by hand:** line saying why and what stands in for it. Then tick it.
  • Nothing gets worse that I know of: not ticked: Risks says a close can take up to 3 s longer, and Measurements shows no close time measured. Measure that cost and state it in Risks, then tick it.
Brought up to the [ready-for-review standard](https://git.magicbean.dev/MagicBeanDev/Longshot/src/branch/main/CONTRIBUTING.md#ready-for-review-out-of-wip) (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know). - *Tested on my own PC*: not ticked: Testing's Tested is game-free only and says the game was not run, though host/ is a game file. Run it yourself on your PC (in the game, for a change the game runs) and say where and what you saw in Testing's *Tested*; or, when the fix's own case can't be run by hand, add a `**Not testable by hand:**` line saying why and what stands in for it. Then tick it. - *Nothing gets worse that I know of*: not ticked: Risks says a close can take up to 3 s longer, and Measurements shows no close time measured. Measure that cost and state it in Risks, then tick it.
Some checks are pending
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 8s
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/close-waits-for-save:TrashPanda-fix/close-waits-for-save'
git switch 'TrashPanda-fix/close-waits-for-save'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1422
No description provided.