MagicBeanDev/Hookshot
Archived
Watch
6
0
Fork
You've already forked Hookshot
 0
Code Issues 125 Pull requests 205 Activity

feat(paths): saves and settings in a per-user data folder, caches and logs in the game folder's cache/ and logs/ #2317

Open
LNDF wants to merge 9 commits from LNDF/structure into main
pull from: LNDF/structure
merge into: MagicBeanDev:main
Conversation 13 Commits 9 Files changed 47 +1367 -485
LNDF commented 2026-11-11 14:48:07 +00:00 (Migrated from github.com)

Saves, settings, logs and several GB of shader caches were all written loose into the game folder (on Windows even the saves), mixed in with what the build made. This sorts them: your saves and settings go to a per-user data folder that survives rebuilding or deleting the game folder, and the caches and logs go into two clearly named folders of the game folder.

What changed

Where files go

What Before Now
Saves (save/, mouse_sens.txt, …) <game>\user (Windows), ~/.local/share/hookshot (Linux) data folder: %LOCALAPPDATA%\Hookshot / $XDG_DATA_HOME/hookshot
hookshot_options.ini, hookshot_env.txt, hookshot_update_skip.txt game folder data folder (hookshot_env.txt beside the game is still read when the data folder has none)
Shader and pipeline caches, hookshot_driver_faults.txt game folder <game>/cache/ (safe to delete; one per game folder, so a release and a nightly don't invalidate each other)
Generated button prompts <game>/prompts/ <game>/cache/prompts/
hookshot_log*.txt, hookshot_startup.log, hookshot_frames.csv game folder <game>/logs/
language.txt, hookshot_shaders.bin, shipped lists game folder unchanged (the build's own)

Exceptions:

  • --user <folder> sets the data folder; a run given an explicit image.bin uses ./user.
  • Portable install (.hookshot_builder_home in the game folder, #2367): the data folder is <game>\user, so saves go with the drive. Only the settings move into it; nothing goes to %LOCALAPPDATA%.
  • A second game folder whose saves clash with the first's (a nightly beside a release) keeps its saves and settings in its own <game>\user.

Code

  • New host/hos_data.c/.h: picks the data folder once at start (before hookshot_env.txt and the log), makes cache/ and logs/, and moves an older build's files there on the first start. Replaces Linux's saves_dir() in hos_main.c.
    • Saves move all or nothing: an entry that can't move brings back the others (a partial original left by a copy is removed first).
    • Links (#2282's handling, carried over): on Linux a link in the saves is made again in the data folder to the same folder, never renamed or copied through; a user that is a link itself is used as it is. On Windows a link or junction keeps the saves where they are.
    • Saves in both places (a downgrade, then the upgrade again): this folder's are used, nothing is merged, and saves-in-two-places.txt says where both are and how to switch.
    • A copied user that can't be fully deleted is renamed user.moved.
    • files-moved.txt says what moved; the log also says when a settings file beside the game is not read.
  • Host readers of the settings (nn_applet, nn_hid, nn_audio, discord_rpc, prompts, hos_main, hos_update) use hos_data_path().
  • Renderer: MkwVkHost.data_dir (MKWVK_API_VERSION 19); vk_paths.h gets data_path(), cache_path(), log_path(), used for the options file, the program cache, pipeline cache, notes, state list, prepared record, driver faults and frame trace.
  • The Vulkan pipeline cache is renamed mkw_vk_pipelines*.bin → hookshot_vk_pipelines*.bin; existing ones are renamed as they move, so nothing is rebuilt.
  • make_game.py seeds the recorded pipeline list into cache/hookshot_pipelines.bin, as main seeds the game folder's.
  • scripts/oot_saves.py finds Windows saves in %LOCALAPPDATA%\Hookshot without --hookshot.
  • Docs: readme (where files go, updating, uninstalling, log location), CONTRIBUTING, development, versioning, saves.md, the game-problem issue form.

Split out at review: the Linux builder's DLSS SDK move is #2485 (merged); its Windows counterpart is #2528.

Testing

  • python -I -B tests/run_all.py on Linux: same results as main. #2282's tests/test_linux_saves_move.py runs all its cases on hos_data.c. New tests/test_data_dir.py builds hos_data.c and covers folder choice, the migration, the bin/ layout, a second game folder, a relative link on the same drive, the portable rule, the "not read" log, and copies across file systems (rollback, links, read-only originals).
  • hookshot_vk.so compiled and linked in an Ubuntu 24.04 container; Linux AppImage built.
  • Not yet done: a Windows build, and a run of the game (updating a folder built by main). The Windows save move across drives is the main thing to check.

🤖 Generated with Claude Code

Saves, settings, logs and several GB of shader caches were all written loose into the game folder (on Windows even the saves), mixed in with what the build made. This sorts them: your saves and settings go to a per-user data folder that survives rebuilding or deleting the game folder, and the caches and logs go into two clearly named folders of the game folder. ## What changed **Where files go** | What | Before | Now | |---|---|---| | Saves (`save/`, `mouse_sens.txt`, …) | `<game>\user` (Windows), `~/.local/share/hookshot` (Linux) | data folder: `%LOCALAPPDATA%\Hookshot` / `$XDG_DATA_HOME/hookshot` | | `hookshot_options.ini`, `hookshot_env.txt`, `hookshot_update_skip.txt` | game folder | data folder (`hookshot_env.txt` beside the game is still read when the data folder has none) | | Shader and pipeline caches, `hookshot_driver_faults.txt` | game folder | `<game>/cache/` (safe to delete; one per game folder, so a release and a nightly don't invalidate each other) | | Generated button prompts | `<game>/prompts/` | `<game>/cache/prompts/` | | `hookshot_log*.txt`, `hookshot_startup.log`, `hookshot_frames.csv` | game folder | `<game>/logs/` | | `language.txt`, `hookshot_shaders.bin`, shipped lists | game folder | unchanged (the build's own) | Exceptions: - `--user <folder>` sets the data folder; a run given an explicit `image.bin` uses `./user`. - **Portable install** (`.hookshot_builder_home` in the game folder, #2367): the data folder is `<game>\user`, so saves go with the drive. Only the settings move into it; nothing goes to `%LOCALAPPDATA%`. - **A second game folder** whose saves clash with the first's (a nightly beside a release) keeps its saves *and* settings in its own `<game>\user`. **Code** - New `host/hos_data.c`/`.h`: picks the data folder once at start (before `hookshot_env.txt` and the log), makes `cache/` and `logs/`, and moves an older build's files there on the first start. Replaces Linux's `saves_dir()` in `hos_main.c`. - Saves move all or nothing: an entry that can't move brings back the others (a partial original left by a copy is removed first). - Links (#2282's handling, carried over): on Linux a link in the saves is made again in the data folder to the same folder, never renamed or copied through; a `user` that is a link itself is used as it is. On Windows a link or junction keeps the saves where they are. - Saves in both places (a downgrade, then the upgrade again): this folder's are used, nothing is merged, and `saves-in-two-places.txt` says where both are and how to switch. - A copied `user` that can't be fully deleted is renamed `user.moved`. - `files-moved.txt` says what moved; the log also says when a settings file beside the game is not read. - Host readers of the settings (`nn_applet`, `nn_hid`, `nn_audio`, `discord_rpc`, `prompts`, `hos_main`, `hos_update`) use `hos_data_path()`. - Renderer: `MkwVkHost.data_dir` (`MKWVK_API_VERSION` **19**); `vk_paths.h` gets `data_path()`, `cache_path()`, `log_path()`, used for the options file, the program cache, pipeline cache, notes, state list, prepared record, driver faults and frame trace. - The Vulkan pipeline cache is renamed `mkw_vk_pipelines*.bin` → `hookshot_vk_pipelines*.bin`; existing ones are renamed as they move, so nothing is rebuilt. - `make_game.py` seeds the recorded pipeline list into `cache/hookshot_pipelines.bin`, as `main` seeds the game folder's. - `scripts/oot_saves.py` finds Windows saves in `%LOCALAPPDATA%\Hookshot` without `--hookshot`. - Docs: readme (where files go, updating, uninstalling, log location), CONTRIBUTING, development, versioning, saves.md, the game-problem issue form. Split out at review: the Linux builder's DLSS SDK move is #2485 (merged); its Windows counterpart is #2528. ## Testing - `python -I -B tests/run_all.py` on Linux: same results as `main`. #2282's `tests/test_linux_saves_move.py` runs all its cases on `hos_data.c`. New `tests/test_data_dir.py` builds `hos_data.c` and covers folder choice, the migration, the `bin/` layout, a second game folder, a relative link on the same drive, the portable rule, the "not read" log, and copies across file systems (rollback, links, read-only originals). - `hookshot_vk.so` compiled and linked in an Ubuntu 24.04 container; Linux AppImage built. - **Not yet done:** a Windows build, and a run of the game (updating a folder built by `main`). The Windows save move across drives is the main thing to check. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
LNDF commented 2026-11-11 15:03:34 +00:00 (Migrated from github.com)

Once we update to a major versioon, we can remove the migration code

Once we update to a major versioon, we can remove the migration code
Arkex-ai commented 2026-11-11 18:04:46 +00:00 (Migrated from github.com)

Heads-up on an integration point with #2367 (portable Windows Builder): it keeps the Builder home, settings/work, updater and game-to-Builder association relocatable. This PR changes the game's save/settings location, so we should agree the behavior when a portable-built game starts: shared per-user data by default versus an explicit game-local --user profile. I don't want to change the default save policy implicitly or migrate existing saves twice. Could we coordinate that contract and the migration tests before either branch absorbs the other's path changes? The launcher proposals #2373 and #2403 also need to consume the same decision.

Heads-up on an integration point with #2367 (portable Windows Builder): it keeps the Builder home, settings/work, updater and game-to-Builder association relocatable. This PR changes the game's save/settings location, so we should agree the behavior when a portable-built game starts: shared per-user data by default versus an explicit game-local `--user` profile. I don't want to change the default save policy implicitly or migrate existing saves twice. Could we coordinate that contract and the migration tests before either branch absorbs the other's path changes? The launcher proposals #2373 and #2403 also need to consume the same decision.
LNDF commented 2026-11-11 19:43:10 +00:00 (Migrated from github.com)

Heads-up on an integration point with #2367 (portable Windows Builder): it keeps the Builder home, settings/work, updater and game-to-Builder association relocatable. This PR changes the game's save/settings location, so we should agree the behavior when a portable-built game starts: shared per-user data by default versus an explicit game-local --user profile. I don't want to change the default save policy implicitly or migrate existing saves twice. Could we coordinate that contract and the migration tests before either branch absorbs the other's path changes? The launcher proposals #2373 and #2403 also need to consume the same decision.

<@324633150510727178> about the comment on #2317 and the portable/launcher PRs. here's what I'm thinking:

  • normal installs: saves + settings go to the per-user data folder (%LOCALAPPDATA%\Hookshot / ~/.local/share/hookshot), caches and logs stay in the game folder (cache/, logs/)
  • portable (#2367): the data folder just stays inside the game folder, so everything moves with the SSD. probably \user since that's where windows saves already are, so nothing has to be migrated. the game can tell it's portable from the .hookshot_builder_home marker
  • the migration to the per-user folder only happens on normal installs, so saves never get moved twice
  • launcher (#2373): anything it writes like hookshot_env.txt goes to the data folder, wherever that ends up, not the game folder

if that sounds ok I'll add the portable part to my PR so you can build on it

> Heads-up on an integration point with [#2367](https://git.magicbean.dev/MagicBeanDev/Hookshot/pulls/2367) (portable Windows Builder): it keeps the Builder home, settings/work, updater and game-to-Builder association relocatable. This PR changes the game's save/settings location, so we should agree the behavior when a portable-built game starts: shared per-user data by default versus an explicit game-local `--user` profile. I don't want to change the default save policy implicitly or migrate existing saves twice. Could we coordinate that contract and the migration tests before either branch absorbs the other's path changes? The launcher proposals [#2373](https://git.magicbean.dev/MagicBeanDev/Hookshot/pulls/2373) and [#2403](https://git.magicbean.dev/MagicBeanDev/Hookshot/pulls/2403) also need to consume the same decision. <@324633150510727178> about the comment on #2317 and the portable/launcher PRs. here's what I'm thinking: - normal installs: saves + settings go to the per-user data folder (%LOCALAPPDATA%\Hookshot / ~/.local/share/hookshot), caches and logs stay in the game folder (cache/, logs/) - portable (#2367): the data folder just stays inside the game folder, so everything moves with the SSD. probably <game>\user since that's where windows saves already are, so nothing has to be migrated. the game can tell it's portable from the .hookshot_builder_home marker - the migration to the per-user folder only happens on normal installs, so saves never get moved twice - launcher (#2373): anything it writes like hookshot_env.txt goes to the data folder, wherever that ends up, not the game folder if that sounds ok I'll add the portable part to my PR so you can build on it
Joey-Magnus (Migrated from github.com) requested changes 2026-11-11 20:08:07 +00:00
Joey-Magnus (Migrated from github.com) left a comment

Separating the player's files from the build's is the right direction, and the save move is careful in the important places: the first pass checks before anything moves, a failure rolls back, delete_tree never enters a link, and user.moved stops a later start from reading leftovers as saves. MkwVkHost.data_dir is added at the end with MKWVK_API_VERSION bumped, and paths stay UTF-8 (GetEnvironmentVariableW + CP_UTF8, under the exe's UTF-8 manifest). I merged the branch with current main (no conflicts) and ran the checks it touches: test_data_dir, test_env_file, test_linux_update_skip, test_oot_saves, test_pipeline_play_cache, test_pipeline_notes_open, test_menu_env_on, test_mods_size_table, test_make_game_ngx, test_utf8_paths, test_prompts_live, the driver-fault and precompile checks and a few more. All pass. I focused on the save and settings migration, since that's where a mistake costs players data. Findings:

blocking: the Windows save move hasn't been run. This PR moves every Windows player's saves for the first time, from <game>\user to %LOCALAPPDATA%\Hookshot, usually across drives, so through copy_tree + delete_tree. Windows is most of the player base, and you list it yourself as not yet done. Please run at least: game folder on a second drive; a save file held open or read-only (to exercise rollback and user.moved); a non-ASCII user name; and a second start afterwards. Also, only the title check has run on 0981ace7. The full game-free suite and the Windows Vulkan build still have to go green on this head.

blocking host/hos_data.c:163-164: move_entry tries MoveFileExA before tree_has_link. On the same drive, an entry that is a symbolic link is therefore renamed as it stands, and a relative one (user/save -> ../../sync/save) names nothing from ~/.local/share/hookshot: the player sees no saves and can't write any. This is the bug #2282 fixes in the code this PR replaces. Checking tree_has_link(from) before the rename (keep the saves where they are, as you already do for the copy) or carrying over #2282's re-linking would close it. Whichever of the two PRs lands second needs to carry the other's handling and its test (test_linux_saves_move.py cuts saves_dir out of hos_main.c, which this PR deletes).

should fix host/hos_data.c:294-305: when a game folder's saves can't move (the usual case for a second game folder, such as a nightly beside a release, whose user/save collides with the first one's), hos_user_dir stays <game>/user, but data_dir is still the shared folder. That folder's hookshot_options.ini also collides, so move_file leaves it loose in the game folder, where nothing reads it any more. That game then runs with its own saves but the other folder's settings, and its own settings are silently dropped. Please either keep saves and settings together (when the saves stay, data_dir stays too) or log clearly that the game folder's hookshot_options.ini is ignored. The same applies to hookshot_env.txt: once the data folder has one, a file put beside the game, which is what issues and Discord have told players to do until now, is ignored without a word (hos_main.c env reading). A log line when both exist would save support time.

should fix: the scope is large for one PR (CONTRIBUTING, "one topic"). The saves/settings move, the cache/ and logs/ split with the pipeline-cache rename, the shipped-notes semantics (seed_pipelines no longer overwrites the notes, notes_adopt_shipped plus .taken), and the Linux builder's NGX SDK relocation can each break separately and would be much easier to review, test and revert apart. At minimum I'd take the shipped-notes change and the NGX SDK move out into their own PRs.

question: the portable-builder contract with #2367/#2373/#2403 (Arkex-ai's comment, your reply) is still open. Merging this first makes "per-user data folder" the default for portable builds too, until the follow-up lands. Is that acceptable to them, or should the portable check (.hookshot_builder_home) be in this PR?

minor host/hos_data.c:168,211: if an entry's copy succeeded but its original couldn't be fully deleted (left), and a later entry fails, the rollback finds the partial original still there (exists(to)) and keeps both. The game then runs from <game>/user with that entry incomplete, while the complete copy sits in the data folder. Nothing is lost, but the player sees missing saves. The rollback could delete the partial original before moving the complete copy back.

minor: the PR description says MKWVK_API_VERSION 16, but the code is 18, and #2261 also takes 18 for MkwVkHost.wheel, so whichever merges second has to go to 19. Also, nx2/make_game.py:1021 says the notes are in "the data folder's shaders/hookshot_pipelines.bin", but pipe_notes_path() uses cache/hookshot_pipelines.bin.

Separating the player's files from the build's is the right direction, and the save move is careful in the important places: the first pass checks before anything moves, a failure rolls back, `delete_tree` never enters a link, and `user.moved` stops a later start from reading leftovers as saves. `MkwVkHost.data_dir` is added at the end with `MKWVK_API_VERSION` bumped, and paths stay UTF-8 (`GetEnvironmentVariableW` + `CP_UTF8`, under the exe's UTF-8 manifest). I merged the branch with current main (no conflicts) and ran the checks it touches: `test_data_dir`, `test_env_file`, `test_linux_update_skip`, `test_oot_saves`, `test_pipeline_play_cache`, `test_pipeline_notes_open`, `test_menu_env_on`, `test_mods_size_table`, `test_make_game_ngx`, `test_utf8_paths`, `test_prompts_live`, the driver-fault and precompile checks and a few more. All pass. I focused on the save and settings migration, since that's where a mistake costs players data. Findings: **blocking**: the Windows save move hasn't been run. This PR moves every Windows player's saves for the first time, from `<game>\user` to `%LOCALAPPDATA%\Hookshot`, usually across drives, so through `copy_tree` + `delete_tree`. Windows is most of the player base, and you list it yourself as not yet done. Please run at least: game folder on a second drive; a save file held open or read-only (to exercise rollback and `user.moved`); a non-ASCII user name; and a second start afterwards. Also, only the title check has run on `0981ace7`. The full game-free suite and the Windows Vulkan build still have to go green on this head. **blocking** `host/hos_data.c:163-164`: `move_entry` tries `MoveFileExA` before `tree_has_link`. On the same drive, an entry that is a symbolic link is therefore renamed as it stands, and a relative one (`user/save -> ../../sync/save`) names nothing from `~/.local/share/hookshot`: the player sees no saves and can't write any. This is the bug #2282 fixes in the code this PR replaces. Checking `tree_has_link(from)` before the rename (keep the saves where they are, as you already do for the copy) or carrying over #2282's re-linking would close it. Whichever of the two PRs lands second needs to carry the other's handling and its test (`test_linux_saves_move.py` cuts `saves_dir` out of `hos_main.c`, which this PR deletes). **should fix** `host/hos_data.c:294-305`: when a game folder's saves can't move (the usual case for a *second* game folder, such as a nightly beside a release, whose `user/save` collides with the first one's), `hos_user_dir` stays `<game>/user`, but `data_dir` is still the shared folder. That folder's `hookshot_options.ini` also collides, so `move_file` leaves it loose in the game folder, where nothing reads it any more. That game then runs with its own saves but the other folder's settings, and its own settings are silently dropped. Please either keep saves and settings together (when the saves stay, `data_dir` stays too) or log clearly that the game folder's `hookshot_options.ini` is ignored. The same applies to `hookshot_env.txt`: once the data folder has one, a file put beside the game, which is what issues and Discord have told players to do until now, is ignored without a word (`hos_main.c` env reading). A log line when both exist would save support time. **should fix**: the scope is large for one PR (CONTRIBUTING, "one topic"). The saves/settings move, the cache/ and logs/ split with the pipeline-cache rename, the shipped-notes semantics (`seed_pipelines` no longer overwrites the notes, `notes_adopt_shipped` plus `.taken`), and the Linux builder's NGX SDK relocation can each break separately and would be much easier to review, test and revert apart. At minimum I'd take the shipped-notes change and the NGX SDK move out into their own PRs. **question**: the portable-builder contract with #2367/#2373/#2403 (Arkex-ai's comment, your reply) is still open. Merging this first makes "per-user data folder" the default for portable builds too, until the follow-up lands. Is that acceptable to them, or should the portable check (`.hookshot_builder_home`) be in this PR? **minor** `host/hos_data.c:168,211`: if an entry's copy succeeded but its original couldn't be fully deleted (`left`), and a *later* entry fails, the rollback finds the partial original still there (`exists(to)`) and keeps both. The game then runs from `<game>/user` with that entry incomplete, while the complete copy sits in the data folder. Nothing is lost, but the player sees missing saves. The rollback could delete the partial original before moving the complete copy back. **minor**: the PR description says `MKWVK_API_VERSION` 16, but the code is 18, and #2261 also takes 18 for `MkwVkHost.wheel`, so whichever merges second has to go to 19. Also, `nx2/make_game.py:1021` says the notes are in "the data folder's shaders/hookshot_pipelines.bin", but `pipe_notes_path()` uses `cache/hookshot_pipelines.bin`.
LNDF commented 2026-11-11 20:46:02 +00:00 (Migrated from github.com)

Thanks for the review! Pushed 469aff4a:

  • Links (blocking): a link or a folder holding one is now never moved, rename included, so the saves stay with it. Added a test with a relative user/save -> ../../synced-saves on the same drive. Whichever of this and #2282 lands second carries the other's test over.
  • Second game folder: when the saves can't move, the data folder stays with them in <game>\user, so the folder keeps its own saves and settings. The log now also says when a hookshot_options.ini or hookshot_env.txt beside the game isn't read.
  • Rollback: a partial original left by a copy is deleted before the full copy comes back. No test for this one: it needs a delete that half-fails and then a later entry that fails, in that order, which depends on the folder's listing order.
  • Portable: added here. With .hookshot_builder_home in the game folder, the data folder is <game>\user, only the settings move into it, and nothing goes to %LOCALAPPDATA%. Same proposal I sent Arkex-ai.
  • Scope: the Linux DLSS SDK move is now #2485. The shipped-notes change is gone instead of split: with caches in the game folder, make_game.py seeds cache/hookshot_pipelines.bin the same way main seeds the game folder.
  • Minor: description updated (API version 18, noted that #2261 will need 19 if it lands second), and the make_game.py docstring fixed.

Still open: the Windows run (second drive, held/read-only save, non-ASCII user name, second start). I'll post results when I have them.

Thanks for the review! Pushed `469aff4a`: - **Links (blocking):** a link or a folder holding one is now never moved, rename included, so the saves stay with it. Added a test with a relative `user/save -> ../../synced-saves` on the same drive. Whichever of this and #2282 lands second carries the other's test over. - **Second game folder:** when the saves can't move, the data folder stays with them in `<game>\user`, so the folder keeps its own saves *and* settings. The log now also says when a `hookshot_options.ini` or `hookshot_env.txt` beside the game isn't read. - **Rollback:** a partial original left by a copy is deleted before the full copy comes back. No test for this one: it needs a delete that half-fails and then a later entry that fails, in that order, which depends on the folder's listing order. - **Portable:** added here. With `.hookshot_builder_home` in the game folder, the data folder is `<game>\user`, only the settings move into it, and nothing goes to `%LOCALAPPDATA%`. Same proposal I sent Arkex-ai. - **Scope:** the Linux DLSS SDK move is now #2485. The shipped-notes change is gone instead of split: with caches in the game folder, `make_game.py` seeds `cache/hookshot_pipelines.bin` the same way `main` seeds the game folder. - **Minor:** description updated (API version 18, noted that #2261 will need 19 if it lands second), and the `make_game.py` docstring fixed. **Still open:** the Windows run (second drive, held/read-only save, non-ASCII user name, second start). I'll post results when I have them.
Joey-Magnus (Migrated from github.com) reviewed 2026-11-11 21:24:21 +00:00
Joey-Magnus (Migrated from github.com) left a comment

The earlier points are addressed in 469aff4a. Two things remain:

  • Downgrade then upgrade. Verified in host/hos_data.c move_saves/migrate: a player who goes back to an older build in the same folder gets a fresh <game>\user\save. On the next upgrade that collides, so all saves stay in <game>\user and the game shows the new ones, while the migrated saves sit unseen in %LOCALAPPDATA%\Hookshot. Nothing is deleted, but files-moved.txt isn't written and only the log mentions it. This needs handling or a release note.
  • The blocking Windows run is still missing: second drive, save held open or read-only, non-ASCII user name, second start.

Coordination:

  • The conflict is only docs/saves.md.
  • MKWVK_API_VERSION 18 is also claimed by #2261, #2459, #2478 and #2479.
  • #2282 should land first, and this PR should carry its link handling into hos_data.c.
  • #2338's hookshot_mirror_skip.txt is written loose in the game folder, and belongs under cache/ once this lands.
The earlier points are addressed in `469aff4a`. Two things remain: - **Downgrade then upgrade.** Verified in `host/hos_data.c` `move_saves`/`migrate`: a player who goes back to an older build in the same folder gets a fresh `<game>\user\save`. On the next upgrade that collides, so all saves stay in `<game>\user` and the game shows the new ones, while the migrated saves sit unseen in `%LOCALAPPDATA%\Hookshot`. Nothing is deleted, but `files-moved.txt` isn't written and only the log mentions it. This needs handling or a release note. - **The blocking Windows run** is still missing: second drive, save held open or read-only, non-ASCII user name, second start. Coordination: - The conflict is only `docs/saves.md`. - `MKWVK_API_VERSION` 18 is also claimed by #2261, #2459, #2478 and #2479. - #2282 should land first, and this PR should carry its link handling into `hos_data.c`. - #2338's `hookshot_mirror_skip.txt` is written loose in the game folder, and belongs under `cache/` once this lands.
LNDF commented 2026-11-11 22:17:20 +00:00 (Migrated from github.com)

Thanks! Pushed 2b32b1c3 (merged with main):

  • Downgrade then upgrade: when both <game>\user and the data folder have saves, the game folder's are still used and nothing is merged or picked, but it now writes saves-in-two-places.txt in the game folder (both paths, and how to switch: rename user to user.old and start again) and logs it. Mentioned in the readme too. Tested, including switching back to the moved saves.
  • #2282: carried into hos_data.c. On Linux a link in the saves is made again in the data folder to the same folder (never renamed or copied through), links to entries that move follow them, and a user that is a link itself is used as it is. On Windows a link or junction still keeps the saves where they are. test_linux_saves_move.py now builds hos_data.c and all its cases pass, the other-drive ones included.
  • Conflict: docs/saves.md fixed in the merge.
  • API version: main took 18 with #1990's pad inputs, so this is 19 now. If another one lands first I'll bump again when I merge.
  • #2338: it's still open, so nothing to change here yet. If it lands first, I'll move hookshot_mirror_skip.txt to cache_path() and add it to the files the first start moves. If this lands first, it should use cache_path() there.
  • Windows license follow-up: #2528 does the same as #2485 for both Windows builders (the WPF one and builder/py).

About the Windows run: that one will have to wait a bit. I don't have a good Windows machine or a second drive to test with right now. I'll post the results (second drive, held/read-only save, non-ASCII user name, second start) once I can, or if someone with that setup can try it before, even better.

Thanks! Pushed `2b32b1c3` (merged with main): - **Downgrade then upgrade:** when both `<game>\user` and the data folder have saves, the game folder's are still used and nothing is merged or picked, but it now writes `saves-in-two-places.txt` in the game folder (both paths, and how to switch: rename `user` to `user.old` and start again) and logs it. Mentioned in the readme too. Tested, including switching back to the moved saves. - **#2282:** carried into `hos_data.c`. On Linux a link in the saves is made again in the data folder to the same folder (never renamed or copied through), links to entries that move follow them, and a `user` that is a link itself is used as it is. On Windows a link or junction still keeps the saves where they are. `test_linux_saves_move.py` now builds `hos_data.c` and all its cases pass, the other-drive ones included. - **Conflict:** `docs/saves.md` fixed in the merge. - **API version:** main took 18 with #1990's pad inputs, so this is 19 now. If another one lands first I'll bump again when I merge. - **#2338:** it's still open, so nothing to change here yet. If it lands first, I'll move `hookshot_mirror_skip.txt` to `cache_path()` and add it to the files the first start moves. If this lands first, it should use `cache_path()` there. - **Windows license follow-up:** #2528 does the same as #2485 for both Windows builders (the WPF one and `builder/py`). About the **Windows run**: that one will have to wait a bit. I don't have a good Windows machine or a second drive to test with right now. I'll post the results (second drive, held/read-only save, non-ASCII user name, second start) once I can, or if someone with that setup can try it before, even better.
hckrman101 commented 2026-11-11 22:55:48 +00:00 (Migrated from github.com)

This now conflicts with main in docs/development.md. Could you merge main into the branch (or rebase) when you get a chance? I'll take another look once it's up to date.

This now conflicts with main in `docs/development.md`. Could you merge main into the branch (or rebase) when you get a chance? I'll take another look once it's up to date.
LNDF commented 2026-11-11 23:28:41 +00:00 (Migrated from github.com)

Merged main (26793b60); docs/development.md was the only conflict. The merge also brought #2447's Save my pipeline list: nx2/pipeline_share.py now reads the list from cache/, the log from logs/ and the options from the data folder (old places as fallback), with a test.

Merged main (`26793b60`); `docs/development.md` was the only conflict. The merge also brought #2447's Save my pipeline list: `nx2/pipeline_share.py` now reads the list from `cache/`, the log from `logs/` and the options from the data folder (old places as fallback), with a test.
Arkex-ai commented 2026-11-12 01:42:43 +00:00 (Migrated from github.com)

One new integration point: #2600 appends romfs_read to MkwVkHost and advances main's API18 to19. This PR appends data_dir as19; #2594 appends take_text as20. Whichever lands second needs the complete callback layout and a version bump from the landed head, including matching host initializers. I'll adapt #2600 if either lands first and rerun its startup/font/filesystem checks. The packed backend uses a separate base-asset callback; your data-location policy remains yours.

One new integration point: #2600 appends `romfs_read` to `MkwVkHost` and advances main's API18 to19. This PR appends `data_dir` as19; #2594 appends `take_text` as20. Whichever lands second needs the complete callback layout and a version bump from the landed head, including matching host initializers. I'll adapt #2600 if either lands first and rerun its startup/font/filesystem checks. The packed backend uses a separate base-asset callback; your data-location policy remains yours.
github-sync commented 2026-11-12 14:42:23 +00:00
Owner

@Joey-Magnus requested changes on GitHub · 2026-11-12 14:32 UTC · original

Most of what I raised is fixed: links are never moved (#2282's handling and test carried in), a second game folder keeps saves and settings together, the rollback clears a partial original first, the portable rule is in, and saves-in-two-places.txt (host/hos_data.c:390) covers the downgrade case. API 20 is right against main's 19.

Two things still block it:

  • CI's Windows part 1 fails on this PR's own tests/test_pipeline_share.py (:413). The test sets XDG_DATA_HOME (:409), but on Windows data_dirs() in nx2/pipeline_share.py reads %LOCALAPPDATA%\Hookshot, so the options aren't found (2048x1152). Set and restore LOCALAPPDATA there too.
  • The Windows run is still missing: game folder on a second drive, a save held open or read-only, a non-ASCII user name, a second start. This moves every Windows player's saves, so I'd want one real run, yours or a tester's.

I merged it into current main (clean) and ran the 20 checks it touches on Linux (test_data_dir, test_linux_saves_move, test_pipeline_share, test_env_file, test_utf8_paths, test_prompt_skip among them): all pass.

> [@Joey-Magnus](https://github.com/Joey-Magnus) requested changes on GitHub · 2026-11-12 14:32 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2317#pullrequestreview-5876209428) Most of what I raised is fixed: links are never moved (#2282's handling and test carried in), a second game folder keeps saves and settings together, the rollback clears a partial original first, the portable rule is in, and `saves-in-two-places.txt` (`host/hos_data.c:390`) covers the downgrade case. API 20 is right against main's 19. Two things still block it: - CI's Windows part 1 fails on this PR's own `tests/test_pipeline_share.py` (`:413`). The test sets `XDG_DATA_HOME` (`:409`), but on Windows `data_dirs()` in `nx2/pipeline_share.py` reads `%LOCALAPPDATA%\Hookshot`, so the options aren't found (2048x1152). Set and restore `LOCALAPPDATA` there too. - The Windows run is still missing: game folder on a second drive, a save held open or read-only, a non-ASCII user name, a second start. This moves every Windows player's saves, so I'd want one real run, yours or a tester's. I merged it into current main (clean) and ran the 20 checks it touches on Linux (`test_data_dir`, `test_linux_saves_move`, `test_pipeline_share`, `test_env_file`, `test_utf8_paths`, `test_prompt_skip` among them): all pass. <!-- github-sync:r:5876209428 -->
github-sync commented 2026-11-12 16:55:02 +00:00
Owner

@Nitnek commented on GitHub · 2026-11-12 16:33 UTC · original

Native Windows migration evidence at 47786aaf7d4dca7e1a54434b187a38775e47e363

I exercised the unchanged host/hos_data.c from this exact head in a small guarded standalone executable using real Win32/CRT calls and the production UTF-8 manifest (GetACP() == 65001). Game-free source qualification only: no game launch, real player saves or installation changes. The PR head still matches at this pre-publication check.

The extended set has 8 migration cases: 5 conforming and 3 functional failures, plus a separate positive notification witness. All payloads are authored synthetic data, at most 88 bytes per case, under isolated fake executable/CWD/LOCALAPPDATA roots on verified distinct volumes. A native exit0 did not establish a complete active data directory.

1. Cross-volume rollback selects a partial source — two cases.

The actual enumeration was a-tree, then z-tree, each holding two files. The first entry copies completely to C:, but deleting its original removes the free file and leaves the other file: one variant uses READONLY, the other a real CreateFileW(GENERIC_READ, FILE_SHARE_READ) handle without DELETE sharing. A separate share-zero handle makes the later entry's copy fail.

Rollback cannot remove the refractory original, so the complete first-entry copy remains on C:. Nevertheless, both data= and user= select D:game/user, now 3/4 files. C: has only the first entry. Neither root alone is complete; their union retains all original sizes/hashes. Recoverability of that union is not a complete active directory or successful all-or-nothing restoration. Relevant paths: hos_data.c:218-221, 278-287, 381-405.

2. A residual source becomes active on the next startup — one case.

Without the later copy failure, the share-READ/no-DELETE case copies 4/4 files to C: and initially selects that complete directory. D: retains one held file and putting user aside fails. After the handle is closed, a fresh second process returns0 but selects D:game/user with 1/4 files, while the full C: copy remains intact. File hashes are unchanged between starts; the selected directory is not. Relevant paths: 289-301, 258-262, 381-405.

The raw harness had called this last case PASS_MIGRATION because it checked first-start completeness and unchanged snapshots. A separate adjudication records the functional FAIL; raw receipts remain unchanged.

Positive controls and verification: share-zero refusal preserves the source payload; same-volume rename can succeed despite a READONLY file; ordinary cross-volume migration and the simple READONLY-copy case remain complete across two processes. A simple same-volume rollback was causally witnessed with real ReadDirectoryChangesW: source REMOVED→ADDED and destination ADDED→REMOVED, alongside an independent add/rename/delete witness. Final hashes alone were not used to infer rollback.

An independent read-only Windows capture matched all 72 native receipts by name, size and SHA256. The parent then replayed the semantic verifier offline and directly cross-compared all72 exports against that capture: the three failures above remain confirmed. Synthetic fixtures/recovery copies are retained; no repair was applied.

Limits: SSH/non-interactive account, not an actually non-ASCII Windows account or Interactive/Limited token; no full Windows CI/test_pipeline_share.py, real save serialization, GPU/gameplay, directory ACL, disk-full/crash or hostile-reparse qualification. Linux/Windows wall clocks were not combined. No binaries or game-derived data were exported. This does not close the outstanding Windows request or the separate LOCALAPPDATA fixture issue.

Could migration keep a complete active directory when rollback or setting aside a residual source fails, rather than returning initialization success with a partial source selected?

AI-assisted evidence analysis and review: Hermes Agent (OpenAI GPT-6.1-sol).

> [@Nitnek](https://github.com/Nitnek) commented on GitHub · 2026-11-12 16:33 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2317#issuecomment-6829571184) ### Native Windows migration evidence at `47786aaf7d4dca7e1a54434b187a38775e47e363` I exercised the **unchanged `host/hos_data.c` from this exact head** in a small guarded standalone executable using real Win32/CRT calls and the production UTF-8 manifest (`GetACP() == 65001`). **Game-free source qualification only: no game launch, real player saves or installation changes.** The PR head still matches at this pre-publication check. The extended set has **8 migration cases: 5 conforming and 3 functional failures**, plus a separate positive notification witness. All payloads are authored synthetic data, at most **88 bytes per case**, under isolated fake executable/CWD/LOCALAPPDATA roots on verified distinct volumes. A native exit0 did not establish a complete active data directory. **1. Cross-volume rollback selects a partial source — two cases.** The actual enumeration was `a-tree`, then `z-tree`, each holding two files. The first entry copies completely to C:, but deleting its original removes the free file and leaves the other file: one variant uses READONLY, the other a real `CreateFileW(GENERIC_READ, FILE_SHARE_READ)` handle without DELETE sharing. A separate share-zero handle makes the later entry's copy fail. Rollback cannot remove the refractory original, so the complete first-entry copy remains on C:. Nevertheless, both `data=` and `user=` select D:`game/user`, now **3/4 files**. C: has only the first entry. Neither root alone is complete; their union retains all original sizes/hashes. **Recoverability of that union is not a complete active directory or successful all-or-nothing restoration.** Relevant paths: `hos_data.c:218-221`, `278-287`, `381-405`. **2. A residual source becomes active on the next startup — one case.** Without the later copy failure, the share-READ/no-DELETE case copies **4/4** files to C: and initially selects that complete directory. D: retains one held file and putting `user` aside fails. After the handle is closed, a **fresh second process** returns0 but selects D:`game/user` with **1/4 files**, while the full C: copy remains intact. File hashes are unchanged between starts; the selected directory is not. Relevant paths: `289-301`, `258-262`, `381-405`. The raw harness had called this last case `PASS_MIGRATION` because it checked first-start completeness and unchanged snapshots. A separate adjudication records the functional FAIL; raw receipts remain unchanged. **Positive controls and verification:** share-zero refusal preserves the source payload; same-volume rename can succeed despite a READONLY file; ordinary cross-volume migration and the simple READONLY-copy case remain complete across two processes. A simple same-volume rollback was causally witnessed with real `ReadDirectoryChangesW`: source REMOVED→ADDED and destination ADDED→REMOVED, alongside an independent add/rename/delete witness. Final hashes alone were not used to infer rollback. An independent read-only Windows capture matched all **72 native receipts** by name, size and SHA256. The parent then replayed the semantic verifier offline and directly cross-compared all72 exports against that capture: the three failures above remain confirmed. Synthetic fixtures/recovery copies are retained; no repair was applied. **Limits:** SSH/non-interactive account, not an actually non-ASCII Windows account or Interactive/Limited token; no full Windows CI/`test_pipeline_share.py`, real save serialization, GPU/gameplay, directory ACL, disk-full/crash or hostile-reparse qualification. Linux/Windows wall clocks were not combined. No binaries or game-derived data were exported. This does not close the outstanding Windows request or the separate LOCALAPPDATA fixture issue. Could migration keep a complete active directory when rollback or setting aside a residual source fails, rather than returning initialization success with a partial source selected? AI-assisted evidence analysis and review: Hermes Agent (OpenAI GPT-6.1-sol). <!-- github-sync:c:6829571184 -->
github-sync commented 2026-11-12 19:01:30 +00:00
Owner

@LNDF commented on GitHub · 2026-11-12 18:20 UTC · original

@Nitnek @Joey-Magnus thanks for the reports. All three cases are fixed in d2b65510:

  • Cases 1 and 2 (rollback onto a partial copy): the move now renames or copies every entry before it deletes any original. If any entry fails, it deletes the copies it made, renames back what it renamed, and keeps using <game>\user. The originals are only deleted after everything got across.
  • Case 3 (a leftover becomes active on the next start): a finished move writes a .hookshot-moved record in the data folder. On later starts, a <game>\user with no file newer than that record is treated as a leftover. It gets set aside as user.moved (or left alone if it's still held) and the data folder stays in use.
  • CI: test_pipeline_share now sets LOCALAPPDATA too, so it no longer reads the runner's real folder on Windows.

tests/test_data_dir.py has two new cases that reproduce them: a copy that fails after other entries moved, and a leftover that can't be deleted on the first start. The old code fails both.

Could you rerun your Windows harness on the new head, Nitnek? And Joey, could you take another look when you have a moment? I still can't do a native run myself (no good Windows PC or second drive), so your results are what I'm going on here.

> [@LNDF](https://github.com/LNDF) commented on GitHub · 2026-11-12 18:20 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2317#issuecomment-6832161942) @Nitnek @Joey-Magnus thanks for the reports. All three cases are fixed in d2b65510: - **Cases 1 and 2 (rollback onto a partial copy):** the move now renames or copies every entry before it deletes any original. If any entry fails, it deletes the copies it made, renames back what it renamed, and keeps using `<game>\user`. The originals are only deleted after everything got across. - **Case 3 (a leftover becomes active on the next start):** a finished move writes a `.hookshot-moved` record in the data folder. On later starts, a `<game>\user` with no file newer than that record is treated as a leftover. It gets set aside as `user.moved` (or left alone if it's still held) and the data folder stays in use. - **CI:** `test_pipeline_share` now sets `LOCALAPPDATA` too, so it no longer reads the runner's real folder on Windows. `tests/test_data_dir.py` has two new cases that reproduce them: a copy that fails after other entries moved, and a leftover that can't be deleted on the first start. The old code fails both. Could you rerun your Windows harness on the new head, Nitnek? And Joey, could you take another look when you have a moment? I still can't do a native run myself (no good Windows PC or second drive), so your results are what I'm going on here. <!-- github-sync:c:6832161942 -->
Commenting is not possible because the repository is archived.
Reviewers
No reviewers
Joey-Magnus
Labels
Clear labels   
accessibility
Barrier affecting people with disabilities

  
autorelease: pending

   
autorelease: tagged

   
bug
Something isn't working

  
dependencies
Pull requests that update a dependency file

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
github_actions
Pull requests that update GitHub Actions code

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
testing wanted

   
wontfix
This will not be worked on

No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
2 participants
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Hookshot!2317
No description provided.