WIP: fix(fs): a find pattern keeps its wildcard at the path buffer's edge #2180
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2180
Loading…
Reference in a new issue
No description provided.
Delete branch "dazemc/Longshot:fix/fs-pattern-buffer"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
A folder whose host path is at the end of the path buffer lists itself instead of its contents:
OpenDirectoryanddelete_treebuilt their find pattern,"dir\*", in a buffer the size of a path (HOS_FS_PATH, 1024). A host path is up to 1023 bytes, so a pattern is two bytes longer than the folder and three with its end: from a 1022-byte folder the wildcard was cut, andFindFirstFileAwas given the folder's own name.OpenDirectorythen answered a listing whose only entry was the folder itself (orfs::ResultPathNotFoundat 1022, where the cut left a trailing separator),GetDirectoryEntryCountcounted the folder, and a walk at 1023 built the child's path back onto the folder and never came back.The pattern now has a buffer with the room for it (
HOS_FS_PATTERN, a path + 4), and both places checksnprintf's answer the way the file already does for a journal's.tmp: a pattern that does not fit isfs::ResultPathNotFound, ashost_pathanswers for a path that does not fit.delete_treealso stops and reports the tree as still there when a child's path does not fit, since a cut path would name the folder itself, not the child. The directory handle keeps the same pattern forGetDirectoryEntryCount(Handle.pat), while a file handle keeps its own path for the trace (Handle.path): oneHandleserves both, so both members exist.To see it: a save or folder path of 1022–1023 bytes.
tests/test_fs_delete_dir.pybuilds exactly those folders; onmain(a2dc5385) its 1023-byte case fails with the folder listed as its own content, and it passes at77fe1c1b.Read first:
host/nn_fs.c—OpenDirectory,GetDirectoryEntryCount,delete_tree,fs_delete_directory_recursively, and theHandlestruct; thentests/test_fs_delete_dir.py.This branch is stacked on #2154 and cannot be reviewed on its own yet. It carries #2154's commit as its base (now
074a38f5, rebased ontomainata2dc5385), so the diff shows both changes until #2154 merges. It is not a mechanical unstack: this fix returns whether the walk went (delete_tree's0x202/0x320002results,gone) and its boundary checks live intests/test_fs_delete_dir.py, both added by #2154; without that commit there is no result for the walk to return and no test file to extend. When #2154 merges, this is rebased ontomain, the duplicate commit drops out, and the suite and the smoke are run again at the new head. The two changes are in one file and are described separately in their commits.For the deletion path this PR supersedes #2154: it carries #2154's result-code change plus the child-path guard, so if that is simpler a reviewer can merge this and close #2154 — nothing in #2154 is lost, and #2154 on its own does not fix the 1023-byte recursion (its body says so).
Testing
Tested: Linux
7.3.4-zen2-2-zen(x86_64), NVIDIA GeForce RTX 4070 Ti, driver620.70.06, Vulkan 1.4. The pack over basea2dc5385at head77fe1c1bcompileshost/nn_fs.cand links (the pack log: 37 s of compiles, link ok). Its runtime boots from the repository savekokiri-start: the Load screen lists the Kokiri Forest save, loads it into Link's house, and Kokiri Forest (child) plays for 15 s —KokiriForest_Child, 3 hearts, 12 life, no fs error in the log — then the game quits with 0; six screenshots and the run's log are in/home/daze/.local/share/longshot-dev/runs/2180-head-smoke(the game's own line:longshot v3.1.1-4-g77fe1c1b (source eb3d815e)). This smoke covers the boot, the save listing/loading and play; the edge itself is covered game-free below, because the save folder's host path here is short.Game-free: on
mainata2dc5385,tests/test_fs_delete_dir.py(with #2154's result-code checks left out so the run reaches the long-path cases) fails at the 1023-byte case's listing (FAIL:533 lists_files(guest): the cut pattern named the folder itself); at9d49c674, the test commit on #2154's head, it fails at the same case (FAIL:558 lists_files(guest)); at77fe1c1bit passes, as doestests/test_fs_links.py.Full suite at
77fe1c1b(tests/run_all.py --strict --jobs 3in an 8 GB scope): 912 passed / 23 failed / 18 skipped in 833 s. The 23 failing names are the same as onmainata2dc5385(measured in #2154's Testing: 911 / 23 / 18 in 635 s), one name at a time:nx2/test_import_windows.py,test_close_waits_for_save,test_compile_zero_byte,test_cpm_discovery,test_damaged_rom_files,test_dev_packs,test_draw_stream_visibility,test_fs_save_commit,test_fsr_uniform_ownership,test_guest_copies,test_hold_toggle,test_lower_atomg_cas,test_menu_i18n,test_menu_keys,test_menu_slider_ends,test_mise_build,test_pads_after_menu,test_recording_admission,test_sim_turf_pools,test_sim_turf_settings,test_switch_table,test_tls_objects,test_vibration_menu. The extra pass is the newtests/test_fs_delete_dir.py. The 833 s is longer than that baseline's 635 s because other jobs ran on the PC beside it; the counts, not the time, are the evidence.Windows:
host/nn_fs.ccompiles as a Windows COFF object with llvm-mingw 20261104'sclang23.1.5 forx86_64-w64-mingw32, with the flagsnx2/build.pygives a host file (--target=x86_64-w64-mingw32 -fms-extensions -D_CRT_SECURE_NO_WARNINGS -march=native -ffp-contract=off -O2 -g0 -mcx16 -DA64_POLLING -DA64_WRITE_TRACK) and headers fromhost/,third_party/a64-kit/rtand the base's generatedgen/. The full Windows game link is not made on this PC.Still to test:
HOS_FS_TRACE=1, start the game from a folder tree (or--userfolder) deep enough that a folder the game opens has a host path of 1022–1023 bytes, and check the game boots and plays and the log'sfs: OpenDirectory(...)lines do not saynot found: onmainthe pattern loses its wildcard there and the folder lists itself. The request is posted after #2154 merges and this branch is rebased ontomain, so the farm runs the final head.Measurements
Nothing to measure: the fix is in host path handling at the end of the path buffer, not a hot path, and it changes nothing per frame.
Risks
A folder or a pattern that does not fit is now refused (
fs::ResultPathNotFound) where before it was silently cut; only paths within three bytes of the 1023-byte limit are affected, and a cut pattern was naming the wrong entry anyway.delete_treenow stops at the first child that does not fit rather than walking past it, so a tree with such a child is reported as still there instead of "deleted". Windows is the target: the sameFindFirstFileA/DeleteFileApath is what players run, and Linux's posix layer reports the same refusals. A revert is enough; no switch, no file format, no save layout changes, and players notice nothing.Checklist
Ready for review
main; the title is a Conventional Commit. (the title is one topic and a Conventional Commit; the branch is stacked on #2154 (its head074a38f5, onmainata2dc5385), so today's diff is two changes, and this is why the PR is WIP. It cannot stand alone: the pattern fix returns whether the walk went (delete_tree's result codes) and its checks live in the test file, both of #2154's commit. Ticks when #2154 merges and this is rebased ontomainwith the stack commit dropped)77fe1c1b, the pack overa2dc5385boots, the Load screen lists the repository save, Kokiri Forest (child) plays 15 s, the game quits with 0; run folder2180-head-smoke.mise run testpasses (any failure that also fails onmainis named): 912 passed / 23 failed / 18 skipped in 833 s at77fe1c1b; the 23 failing names are the same 23mainshows ata2dc5385, listed in Testing.main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
mainand passes here. (tests/test_fs_delete_dir.pyata2dc5385fails at the 1023-byte listing, and at9d49c674on #2154's head; it passes at77fe1c1b.main'sOpenDirectoryanddelete_treeare the same text as9d49c674's before this fix.)tests/that reads changed source is updated, not weakened:tests/test_fs_delete_dir.pygains the 1020/1021/1023-byte cases, andtests/test_fs_links.pygains the pattern's size; no assertion removed.host/nn_fs.cCOFF object with llvm-mingw 20261104 above; Linux is the pack build overa2dc5385at77fe1c1b, which compileshost/nn_fs.cand links, and the game run.Assisted-by:/Co-Authored-By:trailers: all three commits carryAssisted-by: DeepSeek V4.1 Flash 1017 (opencode-go)(git log --format='%(trailers)' 074a38f5..77fe1c1b).Ready to merge
@Beanstalkrequest waits for the rebase ontomainafter #2154 merges, so the farm runs the final head.main, measured again or a comment says why the numbers still hold. On head77fe1c1b: the description check passed; the other 10 jobs areBlocked by required conditionsand are re-confirmed when they report.Tested in game: Linux 7.3.4-zen2, AMD GPU (RADV, Mesa): built the pack over base
9e10f6f7and ran dev.py ab --scene kokiri-forest --runs 1 --seconds 20 (run 20261121-174045-change-kokiri-forest). A main @9e10f6f7: 1817 frames, 90.9 fps, median 10.98 ms, p99 13.71 ms, 1% low 72.9 fps, 0 cold frames. B this PR @88bbb941: 1767 frames, 88.4 fps, median 11.38 ms, p99 14.85 ms, 1% low 67.3 fps, 0 cold frames. Single 20 s runs each side; the 2.5 fps gap is spread, not a claim. Kokiri Forest, quality preset Custom, 67% (1707x960), output 2560x1440, frame generation and vsync off, uncapped.Correction to the 'Tested in game' comment above: it named the wrong GPU. Every run in this session was on this PC's NVIDIA GeForce RTX 4070 Ti, driver 620.70.06, Linux 7.3.4-zen2-2-zen (the game's own line:
vk: device 0: NVIDIA GeForce RTX 4070 Ti, Vulkan 1.4 ... NVIDIA's driver) — not an AMD/RADV/Mesa machine. The numbers are unaffected (run 20261121-174045-change-kokiri-forest): A main @9e10f6f71817 frames, 90.9 fps, median 10.98 ms, p99 13.71 ms, 1% low 72.9 fps, 0 cold frames; B this PR @88bbb9411767 frames, 88.4 fps, 11.38 ms, 14.85 ms, 67.3 fps, 0 cold frames; single 20 s runs each side. The description now states the right hardware.fix(fs): a find pattern keeps its wildcard at the path buffer's edgeto WIP: fix(fs): a find pattern keeps its wildcard at the path buffer's edge88bbb9418477fe1c1be7Rebased onto #2154's rebased head
074a38f5(onmainata2dc5385): the branch is now9d49c674(test),53278a47(pattern fix),77fe1c1b(handle members), andhost/nn_fs.cover074a38f5is the pattern change alone — #2154's result codes are untouched, as the stack note says. The rebase was clean, as expected (git diff 4905e56c 074a38f5 -- host/nn_fs.c tests/test_fs_delete_dir.py tests/test_fs_links.pyis empty).The AI-trailer box was false and is corrected: all three commits now carry
Assisted-by: DeepSeek V4.1 Flash 1017 (opencode-go). The suite and the in-game boxes are unticked until their runs land at this head, and the host-build row and the Ready-to-merge section are added to the description.Game-free at this head:
tests/test_fs_delete_dir.pyfails at9d49c674on the 1023-byte listing (FAIL:558 lists_files(guest)) and passes at77fe1c1b; the same check fails onmainata2dc5385(FAIL:533);tests/test_fs_links.pypasses.host/nn_fs.ccompiles as acoff-x86_64object with llvm-mingw 20261104 / clang 23.1.5. The full suite and the pack's boot and play run at77fe1c1bare running now.Head
77fe1c1bis tested now; the description has the numbers and the two boxes ticked.tests/run_all.py --strict --jobs 3, 8 GB scope) at77fe1c1b: 912 passed / 23 failed / 18 skipped in 833 s. The 23 failing names are the same 23mainshows ata2dc5385(911 / 23 / 18 in 635 s, in #2154's Testing), compared one name at a time; the extra pass is the newtests/test_fs_delete_dir.py. The time is longer only because other jobs ran beside it.77fe1c1b(the pack overa2dc5385, run folder/home/daze/.local/share/longshot-dev/runs/2180-head-smoke): the Load screen lists the Kokiri Forest repository save, loads it into Link's house, and Kokiri Forest (child) plays 15 s (KokiriForest_Child, 3 hearts, 12 life, no fs error in the log); the game quits with 0. The game's own line islongshot v3.1.1-4-g77fe1c1b (source eb3d815e).tests/test_fs_delete_dir.pyfails at9d49c674(the 1023-byte listing,FAIL:558) and onmainata2dc5385(FAIL:533), and passes at77fe1c1bwithtests/test_fs_links.py.host/nn_fs.ccompiles as acoff-x86_64object with llvm-mingw 20261104 / clang 23.1.5. The host-build row is ticked on that and the Linux pack build.Still WIP on one box: the branch is stacked on #2154 (
074a38f5) and is not one topic againstmainuntil that merges. When it does, this is rebased ontomain(the stack commit drops out;git diff 4905e56c 074a38f5over these files is empty), and the suite and the smoke are run again at the new head. The@BeanstalkWindows long-path request is posted after that rebase, so the farm runs the final head.@dazemc only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.