test(profiling): the five checks the profiling changes broke pass again, on Windows too #1323

Open
madebyjordan wants to merge 1 commit from madebyjordan/Longshot:test/profiling-checks into main
Member

Type: test · Risk: low · Tested on: Windows 11, RX 6700 XT · Needs: none

Why

Five checks have failed on main since the profiling changes went in without tests/run_all.py (#1218, the slow-frame recorder and the frame trace's main-thread columns; #1279, HOS_PROFILE=1). Every PR since lists them among the failures it didn't cause. They are test_profiling_mode, test_slow_frames, test_cpu_report, test_vcore_escape and test_invalidate_vcore. Found while testing #1279 on Windows (#1279 (comment)).

What changed

The five checks pass again, on Windows too. Only the tests change: no source does.

  • test_profiling_mode, test_slow_frames:
    • void hos_prof_start(void) is in host/posix/prof.c twice now (a prototype), and void hos_vcore_back(void) is in host/hos_thread.c twice (the stub when vcores are off), so srccut stopped. The markers now take the definition's brace.
    • test_profiling_mode's made-up cpu.csv had one line in the window the report reads (from the first present on), so load.json had no cores. It now has a line a second, as the mode writes them.
  • test_slow_frames on Windows: its driver called setenv, which the Windows C runtime lacks (_putenv_s now, as test_menu_env_overrides does). It also built a 684-byte sample in a 600-byte buffer: strcat wrote past its end, and Windows crashed (0xC0000005).
  • test_cpu_report: scripts/cpu_report.py imports scripts/samples.py since #1279. The check copies the scripts it runs into a folder of its own, and now copies that one too.
  • test_vcore_escape, test_invalidate_vcore: hos_vcore_away / hos_vcore_back read hos_fps_timing, hos_now_ns and the thread's away_at / blocked_total since #1218. Both harnesses now have them, with the Breakdown off.

Measurements

Nothing to measure: tests only.

Flags and settings

None.

Testing

Tested:

  • Each of the five on Windows (llvm-mingw clang): it fails on main @ 6d72ca93 and passes here.
  • python -I -B tests/run_all.py on Windows @ 787c27c5: 593 passed, 37 failed, 90 skipped. Main @ 6d72ca93 on the same PC: 588 passed, 42 failed. The difference is these five and nothing else. The 37 left fail on main too: test_auto_handheld, test_bug_report, test_bug_report_defaults, test_crash_dialog, test_crash_groups, test_crt_options, test_damage_debug_menu, test_display_startup_failure, test_dlss_feature_size, test_failed_compile_keeps_objects, test_framegen_preset, test_guest_imported, test_hardware_report, test_hud_layout_row, test_job_queue, test_linux_preflight, test_linux_pressure, test_mirror_budget, test_missing_nup, test_pads_after_menu, test_pending_guest_import, test_perf_report, test_pipeline_play_cache, test_portable_builder_home, test_portable_builder_startup, test_present_thread, test_readback_timeout_source, test_rumble_menu, test_run_all_changed, test_switch_table, test_target_image_pool, test_tested_configurations, test_upscaler_round_trip, test_upscaler_rows, test_vk_occlusion_queries, test_vk_swapchain_compat, test_walk_key.

Not tested: Linux. These are the same checks, and nothing in the change is Windows-only except the _WIN32 shim for setenv.

Must be tested:

  • Linux, any GPU (anyone, by hand: the five are game-free): python -I -B tests/<each of the five>.py, expecting each to pass

Risks

What could break: nothing the game runs: tests only.

Turning it off: revert the commit.

Compatibility: none.

Issues

Follows #1218 and #1279. Prior work: none (searched hos_prof_start, test_slow_frames, test_cpu_report; other PRs only list them as failing on main).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A check under tests/ that reads changed source updated in this PR, not weakened.

🤖 Generated with Claude Code

**Type:** test · **Risk:** low · **Tested on:** Windows 11, RX 6700 XT · **Needs:** none ## Why Five checks have failed on `main` since the profiling changes went in without `tests/run_all.py` (#1218, the slow-frame recorder and the frame trace's main-thread columns; #1279, `HOS_PROFILE=1`). Every PR since lists them among the failures it didn't cause. They are `test_profiling_mode`, `test_slow_frames`, `test_cpu_report`, `test_vcore_escape` and `test_invalidate_vcore`. Found while testing #1279 on Windows (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1279#issuecomment-40075). ## What changed The five checks pass again, on Windows too. Only the tests change: no source does. - **`test_profiling_mode`, `test_slow_frames`:** - `void hos_prof_start(void)` is in `host/posix/prof.c` twice now (a prototype), and `void hos_vcore_back(void)` is in `host/hos_thread.c` twice (the stub when vcores are off), so srccut stopped. The markers now take the definition's brace. - `test_profiling_mode`'s made-up `cpu.csv` had one line in the window the report reads (from the first present on), so `load.json` had no cores. It now has a line a second, as the mode writes them. - **`test_slow_frames` on Windows:** its driver called `setenv`, which the Windows C runtime lacks (`_putenv_s` now, as `test_menu_env_overrides` does). It also built a 684-byte sample in a 600-byte buffer: `strcat` wrote past its end, and Windows crashed (`0xC0000005`). - **`test_cpu_report`:** `scripts/cpu_report.py` imports `scripts/samples.py` since #1279. The check copies the scripts it runs into a folder of its own, and now copies that one too. - **`test_vcore_escape`, `test_invalidate_vcore`:** `hos_vcore_away` / `hos_vcore_back` read `hos_fps_timing`, `hos_now_ns` and the thread's `away_at` / `blocked_total` since #1218. Both harnesses now have them, with the Breakdown off. ## Measurements Nothing to measure: tests only. ### Flags and settings None. ## Testing **Tested:** - Each of the five on Windows (llvm-mingw clang): it fails on main @ 6d72ca93 and passes here. - `python -I -B tests/run_all.py` on Windows @ 787c27c5: 593 passed, 37 failed, 90 skipped. Main @ 6d72ca93 on the same PC: 588 passed, 42 failed. The difference is these five and nothing else. The 37 left fail on main too: `test_auto_handheld`, `test_bug_report`, `test_bug_report_defaults`, `test_crash_dialog`, `test_crash_groups`, `test_crt_options`, `test_damage_debug_menu`, `test_display_startup_failure`, `test_dlss_feature_size`, `test_failed_compile_keeps_objects`, `test_framegen_preset`, `test_guest_imported`, `test_hardware_report`, `test_hud_layout_row`, `test_job_queue`, `test_linux_preflight`, `test_linux_pressure`, `test_mirror_budget`, `test_missing_nup`, `test_pads_after_menu`, `test_pending_guest_import`, `test_perf_report`, `test_pipeline_play_cache`, `test_portable_builder_home`, `test_portable_builder_startup`, `test_present_thread`, `test_readback_timeout_source`, `test_rumble_menu`, `test_run_all_changed`, `test_switch_table`, `test_target_image_pool`, `test_tested_configurations`, `test_upscaler_round_trip`, `test_upscaler_rows`, `test_vk_occlusion_queries`, `test_vk_swapchain_compat`, `test_walk_key`. **Not tested:** Linux. These are the same checks, and nothing in the change is Windows-only except the `_WIN32` shim for `setenv`. **Must be tested:** - Linux, any GPU (anyone, by hand: the five are game-free): `python -I -B tests/<each of the five>.py`, expecting each to pass ## Risks **What could break:** nothing the game runs: tests only. **Turning it off:** revert the commit. **Compatibility:** none. ## Issues Follows #1218 and #1279. Prior work: none (searched `hos_prof_start`, `test_slow_frames`, `test_cpu_report`; other PRs only list them as failing on main). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
test(profiling): the five checks the profiling changes broke pass again, on Windows too
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
PR description / Description follows the template (pull_request_target) Successful in 9s
CI / Workflow lint (actionlint) (pull_request) Successful in 10s
Guide screenshots / Guide images and local UI source review (pull_request) Successful in 11s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Third-party parts / As their notes say (pull_request) Successful in 14s
787c27c537
#1218 (the slow-frame recorder, the frame trace's main-thread columns) and #1279 (HOS_PROFILE=1) went in without
tests/run_all.py, and five checks failed on main since:

- test_profiling_mode, test_slow_frames: `void hos_prof_start(void)` and `void hos_vcore_back(void)` are each in their
  file twice now (a prototype in host/posix/prof.c; hos_thread.c's stub when vcores are off), so srccut stopped: the
  markers take the definition's brace. test_profiling_mode's made-up cpu.csv had one line in the read window (the
  report reads from the first present on), so load.json had no cores: a line a second now.
- test_slow_frames on Windows: its driver called setenv (none in the Windows C runtime: _putenv_s, as other checks do),
  and built a 684-byte sample in a 600-byte buffer (strcat past its end: Windows crashed).
- test_cpu_report: cpu_report.py imports scripts/samples.py since #1279; the check copies the scripts it runs, and now
  copies that one too.
- test_vcore_escape, test_invalidate_vcore: hos_vcore_away/back read hos_fps_timing, hos_now_ns and the thread's
  away_at / blocked_total since #1218: the harnesses have them, the Breakdown off.

No source changes: the tests only.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Collaborator

Possibly related, for context:

  • PR #1321 fix(profiling): a Windows profile names its C++ code, counts its waits as waits, and gives a thread one row — Overlapping Windows profiling work; also touches profiling reports and tests.
Possibly related, for context: - PR #1321 fix(profiling): a Windows profile names its C++ code, counts its waits as waits, and gives a thread one row — Overlapping Windows profiling work; also touches profiling reports and tests.
Contributor

Tested on Windows 11 at this head (787c27c5, on rc.7's 544229a2), with the Builder's Python and llvm-mingw: test_slow_frames, test_profiling_mode, test_invalidate_vcore, test_vcore_escape and test_cpu_report all pass (5 of 5). On rc.7 itself all five fail.

RELEASING.md's MSVC check, test_invalidate_vcore.py --cxx cl (VS 2026, cl 19.51), also passes here; it fails at rc.7.

Tested on Windows 11 at this head (787c27c5, on rc.7's 544229a2), with the Builder's Python and llvm-mingw: test_slow_frames, test_profiling_mode, test_invalidate_vcore, test_vcore_escape and test_cpu_report all pass (5 of 5). On rc.7 itself all five fail. RELEASING.md's MSVC check, `test_invalidate_vcore.py --cxx cl` (VS 2026, cl 19.51), also passes here; it fails at rc.7.
TrashPanda approved these changes 2026-11-17 10:02:09 +00:00
TrashPanda left a comment

The five profiling checks compile and still check what they checked. I read head 787c27c5. It applies onto current main. This can merge as it is.

The cuts now take hos_prof_start and hos_vcore_back at the definition's brace, so the prototype and the one-line stub no longer stop them. The timed hos_vcore_back is the one that adds blocked_total. The made-up cpu.csv has a line a second, and the report still has to come out at two cores. The slow-frame driver uses _putenv_s on Windows and a buffer large enough for the 684-byte sample. test_cpu_report copies samples.py, which cpu_report.py imports. The two vcore harnesses define the timing fields the thread code reads, with the Breakdown off. Must be tested names a Linux run of the same five checks. Nothing in the change is Windows-only except that setenv shim, and dukenheimer ran all five on this head on Windows, including the MSVC invalidate check. #1321 adds a different profiling test and does not edit these files.

The five profiling checks compile and still check what they checked. I read head `787c27c5`. It applies onto current main. This can merge as it is. The cuts now take `hos_prof_start` and `hos_vcore_back` at the definition's brace, so the prototype and the one-line stub no longer stop them. The timed `hos_vcore_back` is the one that adds `blocked_total`. The made-up `cpu.csv` has a line a second, and the report still has to come out at two cores. The slow-frame driver uses `_putenv_s` on Windows and a buffer large enough for the 684-byte sample. `test_cpu_report` copies `samples.py`, which `cpu_report.py` imports. The two vcore harnesses define the timing fields the thread code reads, with the Breakdown off. Must be tested names a Linux run of the same five checks. Nothing in the change is Windows-only except that `setenv` shim, and dukenheimer ran all five on this head on Windows, including the MSVC invalidate check. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1321 adds a different profiling test and does not edit these files.
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
PR description / Description follows the template (pull_request_target) Successful in 9s
CI / Workflow lint (actionlint) (pull_request) Successful in 10s
Guide screenshots / Guide images and local UI source review (pull_request) Successful in 11s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Third-party parts / As their notes say (pull_request) Successful in 14s
This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'test/profiling-checks:madebyjordan-test/profiling-checks'
git switch 'madebyjordan-test/profiling-checks'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1323
No description provided.