fix(fs): preserve UTF-8 in long-path support #1592

Merged

monsterlag merged 1 commit from fix/long-paths-integration-1497 into fix/long-paths

2026-11-10 10:16:47 +00:00

monsterlag commented

2026-11-10 10:15:17 +00:00

(Migrated from github.com)

The long-path support in #1497 breaks the existing UTF-8 source check and the Windows extraction-boundary fixture. This supplemental PR targets fix/long-paths and fixes those integration failures.

The filesystem wrappers convert explicitly with CP_UTF8, consistent with the game's char strings. The extraction fixture compares equivalent plain and extended Windows paths, then passes the original path to the real metadata call. Existing containment, escape, reparse-point and hard-link assertions remain; a new negative control rejects sibling paths in either spelling.

Validation:

Native Windows fixture compiles the actual fs_wide and fs_find_data helpers with -Wall -Wextra -Werror, without a UTF-8 process manifest. ASCII, Unicode, long drive/UNC paths, and both directory-entry name fields pass on ACP 1252. The exact original helpers from #1497 fail its Unicode check.
Existing UTF-8 manifest/link/startup/no-ANSI source checks pass.
nx2/test_nup_paths.py: all 24 tests pass on Windows and Linux; Linux fixtures use /tmp.
Native checks ran under the workspace disk-reserve guard with 1 GiB estimated peak growth and more than 644 GiB free.
The native fixture is registered with the test runner, uses its compiler handling on Windows, and is explicitly skipped on Linux. Its direct Windows invocation fails clearly when no compiler is available.
One full Windows game-free runner: 209 passed, 9 failed, 22 skipped in 80 seconds, two checks at a time. The changed path/UTF-8 checks pass. Remaining failures are bc_raw_targets, display_startup_failure, fps_overlay_order, fsr3_math, parallel_init_order, parallel_rb_refresh, precompile_compute_guard, present_aspect, and present_semaphores. Their code/fixtures are unchanged by this supplemental patch; the SEH fixture additionally cannot launch the MSVC linker in this local setup. The full suite remains failing.

Hosted Windows Builder packaging passes. The Windows Vulkan and AppImage builds both fail on the base branch's duplicate tx_deadzone in unchanged vk_menu.cpp; upstream already removes that duplicate in #1496. The game-free source workflow targets PRs to main, so it does not run for this feature-branch base. No workflow triggers or checks were changed to hide these results.

Gameplay has not been run for this supplemental patch. The path checks still emit the existing unpack.py:22 unclosed-reader ResourceWarning. No game data or Windows long-path setting changes are involved.

The long-path support in #1497 breaks the existing UTF-8 source check and the Windows extraction-boundary fixture. This supplemental PR targets `fix/long-paths` and fixes those integration failures. The filesystem wrappers convert explicitly with `CP_UTF8`, consistent with the game's char strings. The extraction fixture compares equivalent plain and extended Windows paths, then passes the original path to the real metadata call. Existing containment, escape, reparse-point and hard-link assertions remain; a new negative control rejects sibling paths in either spelling. Validation: - Native Windows fixture compiles the actual `fs_wide` and `fs_find_data` helpers with `-Wall -Wextra -Werror`, without a UTF-8 process manifest. ASCII, Unicode, long drive/UNC paths, and both directory-entry name fields pass on ACP 1252. The exact original helpers from #1497 fail its Unicode check. - Existing UTF-8 manifest/link/startup/no-ANSI source checks pass. - `nx2/test_nup_paths.py`: all 24 tests pass on Windows and Linux; Linux fixtures use `/tmp`. - Native checks ran under the workspace disk-reserve guard with 1 GiB estimated peak growth and more than 644 GiB free. - The native fixture is registered with the test runner, uses its compiler handling on Windows, and is explicitly skipped on Linux. Its direct Windows invocation fails clearly when no compiler is available. - One full Windows game-free runner: 209 passed, 9 failed, 22 skipped in 80 seconds, two checks at a time. The changed path/UTF-8 checks pass. Remaining failures are `bc_raw_targets`, `display_startup_failure`, `fps_overlay_order`, `fsr3_math`, `parallel_init_order`, `parallel_rb_refresh`, `precompile_compute_guard`, `present_aspect`, and `present_semaphores`. Their code/fixtures are unchanged by this supplemental patch; the SEH fixture additionally cannot launch the MSVC linker in this local setup. The full suite remains failing. Hosted Windows Builder packaging passes. The Windows Vulkan and AppImage builds both fail on the base branch's duplicate `tx_deadzone` in unchanged `vk_menu.cpp`; upstream already removes that duplicate in #1496. The game-free source workflow targets PRs to `main`, so it does not run for this feature-branch base. No workflow triggers or checks were changed to hide these results. Gameplay has not been run for this supplemental patch. The path checks still emit the existing `unpack.py:22` unclosed-reader ResourceWarning. No game data or Windows long-path setting changes are involved.