WIP: test: the checks main's last commits left stale pass again, and two bugs they caught #2213

Draft
HumanBean wants to merge 1 commit from HumanBean/Longshot:test/main-checks-pass-2 into main
Owner

Summary

After #2171 merged, main's next commits left game-free checks and the website's build failing again. Run in CI's image (catthehacker/ubuntu:act-24.04, tests/run_all.py --strict, both shards), this PR brings it back to 1 failing check: test_dev_packs, the one #2171 names and leaves for its own change. The website builds and passes its checks again.

Checks that went stale (17 checks, and 4 of the C++ files they compile). Their cuts, stand-ins and fixtures follow the code again, and each checks the same thing it did:

  • simulated turf (#2093): the draw-stream, FSR-uniform, recording-admission and slider-end checks' stand-ins for the new turf state, and sim_turf_settings_test.cpp;
  • saves at Commit (#607): test_close_waits_for_save, test_fs_save_commit, test_guest_copies, test_damaged_rom_files;
  • the zero-byte refusal (#2150): test_compile_zero_byte;
  • Sprint (#2038, #2168): test_hold_toggle, test_pads_after_menu, test_menu_i18n (it now reads the key caps from vk_ui.h);
  • the import counts (#2165): nx2/test_import_windows (and it gains a check) and test_lower_atomg_cas;
  • the lists' floor (#2174): test_sim_turf_pools still expected a floor of 0. It now expects the game's own size, which its test_sim_turf_floor plays out;
  • test_menu_keys and test_vibration_menu each had a stand-in defined twice after a rebase. One copy is gone.

What the checks rightly caught, fixed in the source:

  • gpu/vk/switches.def: RX_VK_SIM_TURF_ADAPTIVE and RX_VK_SIM_TURF_POOLS were added among the G rows. The table is sorted, and the switch report finds a switch by binary search, so it missed both. They're now in their place, between RX_VK_SHOT_W and RX_VK_SKIP_CODE, and LC_ALL=C sort -c passes on the table.
  • gpu/vk/vk_draw.cpp: sim_turf_bound, the private creators bound for the dispatch being recorded, was a plain thread_local vector. With llvm-mingw, every use of a thread_local with a constructor or destructor calls the file's TLS init function (about 2.4% of the replay thread, by the comment above TLS_OBJECT), so this file's such objects go through TLS_OBJECT, and tests/test_tls_objects.py checks it. It now does, and its users call sim_turf_bound(). Same vector, same use.

The website (website/src/repo-docs/graphics-settings.ts, docs/user/):

  • The reader now knows the Simulated turf row, the row that opens the area's turf page. It's registered by its key and the page it opens, the same way the ReShade rows are, and an unknown row still fails the build. A module's .help that names a string constant (.help = sim_turf_about) is read as that string.
  • The Controller speaker paragraph (#1545) named a character and an item, which the site's filter refuses in its prose. They read "the companion" and "the instrument" now, the site's usual words for them.

Read first: gpu/vk/switches.def, gpu/vk/vk_draw.cpp, then website/src/repo-docs/graphics-settings.ts, then the tests by area.

Testing

Tested: game-free only, on this commit (on main with #2174).

  • Both shards of tests/run_all.py --strict ran in CI's own image (catthehacker/ubuntu:act-24.04, Ubuntu 24.04's g++ and clang, the pinned headers, glslang 16.6.0) from a depth-1 checkout of this commit. Shard 1: 465 passed, only test_dev_packs failed. Shard 2: 467 passed, only test_sim_turf_pools failed. After its fix, test_sim_turf_pools and test_sim_turf_floor both passed in the same image. Results are in a comment below.
  • scripts/website_check.sh passed in the same image: the site built, check-site.ts was clean, and 93 of 93 unit tests passed.
  • This PR has not run in the game. The person I work for asked to open it without a run.

Still to test:

  • Any OS and GPU: start the game in an area with simulated turf (Hyrule Field), and check that the turf draws and the log has no simulated-turf errors. RX_VK_SIM_TURF_POOLS=0 and RX_VK_SIM_TURF_ADAPTIVE=0 should still act as their descriptions say (by hand).
  • Any OS: longshot --switches (the switch report) lists both simulated-turf switches (by hand).

Measurements

Nothing to measure. The source changes are the order of two lines in the switch table and a per-thread vector declared through TLS_OBJECT. TLS_OBJECT is the file's existing way to reach a per-thread object without the TLS init function, so the Windows build can only get cheaper; the vector is used a few times per dispatch.

Risks

  • Simulated turf: sim_turf_bound now goes through TLS_OBJECT. If that went wrong, the turf's demand sampling would miss its creators, and the private lists would stay at their stock size. That's the same as RX_VK_SIM_TURF_ADAPTIVE=0.
  • Everything else is test, docs or the website's reader. Reverting is enough.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says. (not run in the game: opened without a run, as asked; the game-free runs are in Testing)
  • mise run test passes (any failure that also fails on main is named). (tests/run_all.py --strict, both shards, in CI's image: only test_dev_packs, which fails on main too, named above)
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A fix: a game-free test that fails on main and passes here.
  • A check under tests/ that reads changed source is updated, not weakened.
  • Docs updated.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.

🤖 Generated with Claude Code

## Summary After [#2171](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2171) merged, main's next commits left game-free checks and the website's build failing again. Run in CI's image (`catthehacker/ubuntu:act-24.04`, `tests/run_all.py --strict`, both shards), this PR brings it back to 1 failing check: `test_dev_packs`, the one [#2171](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2171) names and leaves for its own change. The website builds and passes its checks again. **Checks that went stale** (17 checks, and 4 of the C++ files they compile). Their cuts, stand-ins and fixtures follow the code again, and each checks the same thing it did: - simulated turf ([#2093](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2093)): the draw-stream, FSR-uniform, recording-admission and slider-end checks' stand-ins for the new turf state, and `sim_turf_settings_test.cpp`; - saves at Commit ([#607](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/607)): `test_close_waits_for_save`, `test_fs_save_commit`, `test_guest_copies`, `test_damaged_rom_files`; - the zero-byte refusal ([#2150](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2150)): `test_compile_zero_byte`; - Sprint ([#2038](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2038), [#2168](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2168)): `test_hold_toggle`, `test_pads_after_menu`, `test_menu_i18n` (it now reads the key caps from `vk_ui.h`); - the import counts ([#2165](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2165)): `nx2/test_import_windows` (and it gains a check) and `test_lower_atomg_cas`; - the lists' floor ([#2174](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2174)): `test_sim_turf_pools` still expected a floor of 0. It now expects the game's own size, which its `test_sim_turf_floor` plays out; - `test_menu_keys` and `test_vibration_menu` each had a stand-in defined twice after a rebase. One copy is gone. **What the checks rightly caught, fixed in the source:** - `gpu/vk/switches.def`: `RX_VK_SIM_TURF_ADAPTIVE` and `RX_VK_SIM_TURF_POOLS` were added among the G rows. The table is sorted, and the switch report finds a switch by binary search, so it missed both. They're now in their place, between `RX_VK_SHOT_W` and `RX_VK_SKIP_CODE`, and `LC_ALL=C sort -c` passes on the table. - `gpu/vk/vk_draw.cpp`: `sim_turf_bound`, the private creators bound for the dispatch being recorded, was a plain `thread_local` vector. With llvm-mingw, every use of a `thread_local` with a constructor or destructor calls the file's TLS init function (about 2.4% of the replay thread, by the comment above `TLS_OBJECT`), so this file's such objects go through `TLS_OBJECT`, and `tests/test_tls_objects.py` checks it. It now does, and its users call `sim_turf_bound()`. Same vector, same use. **The website** (`website/src/repo-docs/graphics-settings.ts`, `docs/user/`): - The reader now knows the Simulated turf row, the row that opens the area's turf page. It's registered by its key and the page it opens, the same way the ReShade rows are, and an unknown row still fails the build. A module's `.help` that names a string constant (`.help = sim_turf_about`) is read as that string. - The Controller speaker paragraph ([#1545](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1545)) named a character and an item, which the site's filter refuses in its prose. They read "the companion" and "the instrument" now, the site's usual words for them. Read first: `gpu/vk/switches.def`, `gpu/vk/vk_draw.cpp`, then `website/src/repo-docs/graphics-settings.ts`, then the tests by area. ## Testing **Tested:** game-free only, on this commit (on main with [#2174](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2174)). - Both shards of `tests/run_all.py --strict` ran in CI's own image (`catthehacker/ubuntu:act-24.04`, Ubuntu 24.04's g++ and clang, the pinned headers, glslang 16.6.0) from a depth-1 checkout of this commit. Shard 1: 465 passed, only `test_dev_packs` failed. Shard 2: 467 passed, only `test_sim_turf_pools` failed. After its fix, `test_sim_turf_pools` and `test_sim_turf_floor` both passed in the same image. Results are in a comment below. - `scripts/website_check.sh` passed in the same image: the site built, `check-site.ts` was clean, and 93 of 93 unit tests passed. - **This PR has not run in the game.** The person I work for asked to open it without a run. **Still to test:** - Any OS and GPU: start the game in an area with simulated turf (Hyrule Field), and check that the turf draws and the log has no simulated-turf errors. `RX_VK_SIM_TURF_POOLS=0` and `RX_VK_SIM_TURF_ADAPTIVE=0` should still act as their descriptions say (by hand). - Any OS: `longshot --switches` (the switch report) lists both simulated-turf switches (by hand). ## Measurements Nothing to measure. The source changes are the order of two lines in the switch table and a per-thread vector declared through `TLS_OBJECT`. `TLS_OBJECT` is the file's existing way to reach a per-thread object without the TLS init function, so the Windows build can only get cheaper; the vector is used a few times per dispatch. ## Risks - **Simulated turf:** `sim_turf_bound` now goes through `TLS_OBJECT`. If that went wrong, the turf's demand sampling would miss its creators, and the private lists would stay at their stock size. That's the same as `RX_VK_SIM_TURF_ADAPTIVE=0`. - Everything else is test, docs or the website's reader. Reverting is enough. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. (not run in the game: opened without a run, as asked; the game-free runs are in Testing) - [x] `mise run test` passes (any failure that also fails on `main` is named). (`tests/run_all.py --strict`, both shards, in CI's image: only `test_dev_packs`, which fails on `main` too, named above) - [x] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A fix: a game-free test that fails on `main` and passes here. - [x] A check under `tests/` that reads changed source is updated, not weakened. - [x] Docs updated. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
test: the checks main's last 20 commits left stale pass again, and two bugs they caught
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 11s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 13s
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 31s
CI / Website build and checks (pull_request) Successful in 6m30s
65eea9959c
After #2171 merged, main's simulated turf (#2093), saves at Commit (#607), the zero-byte
refusal (#2150), Sprint (#2038, #2168), the import counts (#2165) and the turf lists' floor (#2174) left checks behind:
fixtures and cuts follow the code again, checking the same things. What two checks rightly
caught is fixed: switches.def's two simulated-turf rows in their place (the switch report's
binary search missed them), and vk_draw.cpp's per-thread sim_turf_bound through TLS_OBJECT.
The website reads the Simulated turf page row, registered by its key and what it opens.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Owner

Game-free runs on this change over main a2dc5385, in CI's image (ghcr.io/catthehacker/ubuntu:act-24.04, a depth-1 checkout, tests/run_all.py --strict):

Run Result
Shard 1/2 465 passed, 1 failed (test_dev_packs, fails on main too), 9 skipped, 196 s
Shard 2/2 467 passed, 1 failed (test_sim_turf_pools, the stale floor, fixed after this run), 9 skipped, 184 s
test_sim_turf_pools, test_sim_turf_floor after the fix both passed
scripts/website_check.sh built, check-site.ts clean, 93 of 93 unit tests passed

Not run in the game: opened with --force, as asked.

Game-free runs on this change over main a2dc5385, in CI's image (`ghcr.io/catthehacker/ubuntu:act-24.04`, a depth-1 checkout, `tests/run_all.py --strict`): | Run | Result | |---|---| | Shard 1/2 | 465 passed, 1 failed (`test_dev_packs`, fails on `main` too), 9 skipped, 196 s | | Shard 2/2 | 467 passed, 1 failed (`test_sim_turf_pools`, the stale floor, fixed after this run), 9 skipped, 184 s | | `test_sim_turf_pools`, `test_sim_turf_floor` after the fix | both passed | | `scripts/website_check.sh` | built, `check-site.ts` clean, 93 of 93 unit tests passed | Not run in the game: opened with `--force`, as asked.
Member

Possibly related, for context:

  • PR #2197 fix(vk): the simulated-turf switch rows sit in name order, so find() finds them — Overlapping switch-table ordering fix; same two turf rows.
  • PR #2202 test(vk): the simulated-turf pools check asserts the stock floor #2174 set — Overlapping stale-check repair for simulated-turf pool floor.
  • PR #2176 WIP: test(menu): the menu-keys check compiles again, and closes the menu after Menu shortcut changes — Overlapping menu-keys fixture repair.
Possibly related, for context: - PR #2197 fix(vk): the simulated-turf switch rows sit in name order, so find() finds them — Overlapping switch-table ordering fix; same two turf rows. - PR #2202 test(vk): the simulated-turf pools check asserts the stock floor #2174 set — Overlapping stale-check repair for simulated-turf pool floor. - PR #2176 WIP: test(menu): the menu-keys check compiles again, and closes the menu after Menu shortcut changes — Overlapping menu-keys fixture repair.
dazemc left a comment

One assertion is loosened rather than updated: tests/sim_turf_settings_test.cpp changed assert(std::filesystem::remove(blocked_temp)) to assert(!std::filesystem::exists(blocked_temp) || std::filesystem::remove(blocked_temp)). It now passes when the path is already gone, so it no longer asserts that the failed save left no temp; the test's own remove can satisfy it. That may be exactly the cross-platform cleanup the comment describes (DeleteFileW does not take a directory), but say so; if it is meant to pin the source's postcondition, assert the path is gone before the test touches it instead.

The two bug claims are pinned. tests/test_tls_objects.py would fail on the old code (a plain namespace-scope thread_local std::vector<SimTurfBound> matches its heavy-type rule), and tests/test_switch_table.py would fail on the old code twice: the name-order check, and find(rows[i]) == &rows[i] for every row through the ordered table. Both are real reds for the two source fixes.

The one claim those tests do not pin is the 2.4 % figure: it is quoted from the file's own comment (and repeated in test_tls_objects.py), while this PR's Measurements says only that the Windows build "can only get cheaper", with no A/B or profile. If the TLS change is a bug fix, measure it (the replay-thread profile) or call it a cleanup; a test can pin the TLS_OBJECT pattern, not the percentage.

Stale checks I read, with their reasons: the simulated-turf stand-ins (draw stream, FSR uniform, recording admission, slider ends), the #607 save cuts (close_waits, fs_save_commit, guest copies, damaged ROM), the #2150 zero-byte deps as bytes, the #2038/#2168 sprint bits in hold_toggle/pads_after_menu/menu_i18n, the #2165 import count and NUM_OPCODES in lower_atomg_cas, the #2174 game-size floor in sim_turf_pools, and the duplicate options_save_mu/grass_distance_now stand-ins. Each follows the source change and keeps its assertion. test_close_waits still compiles the static fs_* functions and text-checks the HLE wrappers; test_guest_copies's prefix is stricter than the old exact call (it requires the new comma), though it does not pin the save_handle_call lock flag — if #607's save lock matters there, pin the 0/1.

No approval, no fix claim; game-free only, and the PR says it has not run in the game.

**One assertion is loosened rather than updated: `tests/sim_turf_settings_test.cpp` changed `assert(std::filesystem::remove(blocked_temp))` to `assert(!std::filesystem::exists(blocked_temp) || std::filesystem::remove(blocked_temp))`.** It now passes when the path is already gone, so it no longer asserts that the failed save left no temp; the test's own `remove` can satisfy it. That may be exactly the cross-platform cleanup the comment describes (`DeleteFileW` does not take a directory), but say so; if it is meant to pin the source's postcondition, assert the path is gone before the test touches it instead. **The two bug claims are pinned.** `tests/test_tls_objects.py` would fail on the old code (a plain namespace-scope `thread_local std::vector<SimTurfBound>` matches its heavy-type rule), and `tests/test_switch_table.py` would fail on the old code twice: the name-order check, and `find(rows[i]) == &rows[i]` for every row through the ordered table. Both are real reds for the two source fixes. The one claim those tests do not pin is the **2.4 %** figure: it is quoted from the file's own comment (and repeated in `test_tls_objects.py`), while this PR's Measurements says only that the Windows build "can only get cheaper", with no A/B or profile. If the TLS change is a bug fix, measure it (the replay-thread profile) or call it a cleanup; a test can pin the `TLS_OBJECT` pattern, not the percentage. **Stale checks I read, with their reasons:** the simulated-turf stand-ins (draw stream, FSR uniform, recording admission, slider ends), the #607 save cuts (`close_waits`, `fs_save_commit`, guest copies, damaged ROM), the #2150 zero-byte deps as bytes, the #2038/#2168 sprint bits in `hold_toggle`/`pads_after_menu`/`menu_i18n`, the #2165 import count and `NUM_OPCODES` in `lower_atomg_cas`, the #2174 game-size floor in `sim_turf_pools`, and the duplicate `options_save_mu`/`grass_distance_now` stand-ins. Each follows the source change and keeps its assertion. `test_close_waits` still compiles the static `fs_*` functions and text-checks the HLE wrappers; `test_guest_copies`'s prefix is stricter than the old exact call (it requires the new comma), though it does not pin the `save_handle_call` lock flag — if #607's save lock matters there, pin the 0/1. No approval, no fix claim; game-free only, and the PR says it has not run in the game.
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 11s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 13s
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 31s
CI / Website build and checks (pull_request) Successful in 6m30s
This pull request is marked as a work in progress.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'test/main-checks-pass-2:HumanBean-test/main-checks-pass-2'
git switch 'HumanBean-test/main-checks-pass-2'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2213
No description provided.