fix(queue): on Windows the job queue waits for a player and sees builds outside it, as on Linux #1523

Merged
HumanBean merged 2 commits from dukenheimer/Longshot:fix/job-queue-windows-player into main 2026-11-19 07:35:04 +00:00
Contributor

Type: fix · Risk: low · Tested on: Windows 11, RTX 5090 (Python 3.14); Linux (WSL Ubuntu 24.04, Python 3.12) · Needs: none

TL;DR

On Windows, the PC's job queue never noticed someone playing. A timed measurement or a "wait until idle" job could start
while the game was running and spoil both, and status always said nobody was playing. The queue now spots a
running game on Windows the way it already does on Linux, and also sees builds started outside it. New tests fail on
main on Windows and Linux and pass here, including real processes on Windows.

Breakdown

  • Who it helps: anyone sharing a Windows PC with agents or scripts that run through the job queue, and the measurements made there.
  • What was wrong: on Windows the queue's player check returned "nobody" without looking, and its check for builds outside the queue was skipped.
  • What changes: on Windows the queue reads the process list and each game's environment, as Linux reads /proc. A game not started by a queued job holds timed and --when-idle jobs, and status names it.
  • What does not change: Linux's behaviour (same results before and after), the queue's rules, slots and order, and anything in the game.
  • How it was checked: new tests over a made-up process table fail on main on both OSes and pass here; real longshot.exe processes on Windows are told apart by LONGSHOT_JOB; full suites on both OSes show no new failure.
  • What is still open: nothing.
  • Risk: low: only the queue's script, and the Windows side only reads processes.

Why

AGENTS.md and the measure skill send every game run, measurement and profile on a PC through scripts/job_queue.py,
so that nothing runs over a person playing or over another measurement. On Windows that protection was missing. At
main 310e7c9d:

  • user_playing() returns False whenever WINDOWS (the check before its game_procs call);
  • status sets games = [] and heavy = [] on Windows, so it always prints playing: nobody and never a build outside the queue;
  • game_procs, heavy_procs, argv_of, env_of and tree read /proc only. On Windows they find nothing, so a timed run never waits for a build outside the queue either.

So a timing/profile job or a --when-idle ticket on Windows starts while someone plays, and its numbers are
measured over the game. The PC's release-check runner found this.

Prior work: none found. I searched job_queue, user_playing, game_procs, heavy_procs, "player protection", "playing: nobody",
"you are playing" and the team chat. #1315 (NitnekLeBorgne) changes only the queue's history. #1085 and #1100 built the
queue and its Windows job objects, and neither covers this.

How to reproduce

Seen on: Windows 11, RTX 5090, main 310e7c9d
Settings and switches: defaults
Steps: Start the game (longshot.exe), then run python -I scripts/job_queue.py status, or queue python -I scripts/job_queue.py run fps --mode timing -- <anything>. Game-free: python -I -B tests/test_job_queue.py from this PR's test commit.
Expected: playing: longshot.exe (pid N), and the timed job waits: "fps waits: you are playing"
Actual: playing: nobody, and the timed job starts at once

What changed

The queue's process reads now work on Windows, so the player check, status and the stray-build check work there as
they do on Linux.

  • queue: scripts/job_queue.py:
    • _win_snapshot() lists every process with its parent and program (Toolhelp).
    • _win_params() reads a process's command line and environment from its own PEB (NtQueryInformationProcess, ReadProcessMemory), as /proc/<pid>/cmdline and environ. It only reads a 64-bit process the user may read; anything else is unreadable, as on Linux.
    • all_pids, the new exes/exe_of/comm_of, argv_of, env_of (Windows' =C: entries are no variables) and tree (by parent, a child born after its parent) use these on Windows.
    • game_procs and heavy_procs first match the program's name, so only games' and pythons' memory is read. They also accept backslashed script paths.
    • user_playing() and status no longer skip Windows.
  • docs: docs/development.md ("On Windows too") and the script's usage say how a player is seen.
  • tests: tests/test_job_queue.py (first commit) adds two classes:
    • WindowsPlayers runs the Windows code over a made-up process table on any OS. It covers a player, a job's game, shader preparation, an unreadable game, builds outside the queue with backslashes, a job's tree and a reused pid.
    • WindowsPlayersReal (Windows only) uses a copy of ping.exe named longshot.exe, with and without LONGSHOT_JOB, and a python running nx2/build.py.

Measurements

Verdict: fixed. On main the new tests fail on Windows (7) and Linux (5, the real-process ones being Windows-only); here they all pass. Linux's own detection is unchanged.

Hardware: Windows 11, RTX 5090, Python 3.14; WSL Ubuntu 24.04, Python 3.12
Scene: none: game-free checks
Method: python -I -B tests/test_job_queue.py -v on main (with this PR's test) and on this PR, on Windows and Linux. The fix was then broken on purpose and the tests run again. On Linux, a copy of sleep named longshot was run with and without LONGSHOT_JOB through game_procs/user_playing, on main and here.
How to reproduce: the commands above. Rebased since onto main 4b68c42b (its four commits touch only docs/development.md elsewhere among these files); test_job_queue.py gives the same result there. Rebased again onto main 1055077d (head f8b14525; main changed neither scripts/job_queue.py nor tests/test_job_queue.py): this PR

Before: main @ 310e7c9d (with the test: 93f9a3b8) After: this PR @ 716cccd3 (64d0afbc on 4b68c42b; f8b14525 on 1055077d)
new tests, Windows 7 of 7 fail (1 error) 7 of 7 pass
new tests, Linux 5 of 5 fail (2 skipped: Windows only) 5 of 5 pass
Linux real game, no job / under a job yours / probe, playing: True yours / probe, playing: True
status on this PC, a build running beside it under pc-lock (nothing: skipped on Windows) (a build outside the queue: pid 181368 ...python.exe -I nx2\make_game...)
Before: 93f9a3b8 (main 310e7c9d + the test), Windows
FAIL: test_games: AssertionError: {} != {40: '', 41: 'fps', 42: 'prepare', 43: None}
FAIL: test_playing: AssertionError: False is not true : a longshot.exe of nobody's job: someone is playing
FAIL: test_status: 'playing: longshot.exe (pid 40)' not found in 'playing: nobody\n...'
FAIL: test_builds_outside: Lists differ: [] != [50, 53]
FAIL: test_tree: Lists differ: [60] != [53, 60, 61]
FAIL: test_a_build_outside_the_queue: AssertionError: 509708 not found in []
ERROR: test_a_game_and_a_jobs_game
Ran 19 tests  FAILED (failures=7, errors=1, skipped=1)   (one of the 7: test_ram_file_system, failing on main on Windows already)

After: 716cccd3, Windows
test_builds_outside ... ok / test_games ... ok / test_playing ... ok / test_status ... ok / test_tree ... ok
test_a_build_outside_the_queue ... ok / test_a_game_and_a_jobs_game ... ok
(test_ram_file_system still FAIL: as on main)

After, Linux: Ran 19 tests  OK (skipped=5)

Broken on purpose (Windows): main's `if WINDOWS` put back in user_playing: test_playing and test_a_game_and_a_jobs_game fail;
the environment's "=C:" entries kept: test_games fails

Flags and settings

None: no switch is added or changed. LONGSHOT_QUEUE_GAMES and LONGSHOT_QUEUE_HEAVY now apply on Windows too, with
their defaults (longshot.exe is already in the games list).

Testing

Tested:

  • tests/test_job_queue.py on my own PC (Windows 11), on main and here, and in WSL Ubuntu 24.04, on main and here (table above). The real-process tests ran on Windows: a longshot.exe without LONGSHOT_JOB is a player, one with LONGSHOT_JOB=probe is that job's, its arguments are read, and a python running nx2/build.py is a build outside the queue unless it's in the caller's tree.
  • python -I scripts/job_queue.py status on this PC: playing: nobody (no game running) in 0.39 s, and it named the release tester's make_game.py running beside it as a build outside the queue.
  • python -I -B tests/run_all.py, this PR @ 64d0afbc and main @ 4b68c42b in a clean checkout, the same runs. Windows 11 (12 jobs): this PR 665 passed, 50 failed; main 664 passed, 51 failed. test_longshot_ctl failed only on main, a timeout it sometimes has under load, not this PR's. Linux, WSL Ubuntu 24.04 (8 jobs): 594 passed, 50 failed on both. None fails here and not on main. test_job_queue fails on Windows on both, for its test_ram_file_system alone (its new cases pass here), and passes on Linux on both. The 50 that fail on both, on this PC: Windows test_auto_handheld, test_bug_report_defaults, test_build_long_paths, test_compute_warm_apart, test_cpu_report, test_crash_dialog, test_crt_options, test_damage_debug_menu, test_dev_packs, test_discover, test_display_startup_failure, test_dlss_feature_size, test_draw_stream_visibility, test_failed_compile_keeps_objects, test_fsr_image_retirement, test_fsr_uniform_ownership, test_guest_imported, test_hud_layout_row, test_import_reach, test_invalidate_quick, test_invalidate_vcore, test_job_queue, test_libc_hooks, test_lift_lookups, test_linux_preflight, test_linux_pressure, test_mirror_budget, test_missing_nup, test_pads_after_menu, test_parallel_init_order, test_pending_guest_import, test_pipeline_play_cache, test_present_thread, test_profiling_mode, test_psychov35_fsr3_fg, test_readback_timeout_source, test_recorder_sync, test_replay_write_order, test_run_all_changed, test_sdiv_overflow, test_slow_frames, test_staging_burst, test_switch_table, test_target_image_pool, test_tested_configurations, test_upscaler_rows, test_vcore_escape, test_vk_occlusion_queries, test_walk, test_walk_key; Linux byte_compile_tracked, test_bake_ds, test_bug_report, test_bug_report_defaults, test_check_third_party, test_control, test_cpu_report, test_crash_groups, test_crt_options, test_crt_shaders, test_damage_debug_menu, test_dev_packs, test_display_startup_failure, test_dlss_feature_size, test_failed_compile_keeps_objects, test_finish_output, test_guest_imported, test_hardware_report, test_hud_layout_row, test_invalidate_quick, test_invalidate_vcore, test_libc_hooks, test_link_skip, test_linux_downgrade, test_linux_pressure, test_linux_update, test_log_open_failure, test_mirror_budget, test_missing_nup, test_mods_size_table, test_oot_state_tap, test_pending_guest_import, test_pipeline_cache_file, test_pipeline_cache_save, test_pipeline_play_cache, test_pipeline_tour, test_present_thread, test_profiling_mode, test_psychov35_fsr3_fg, test_readback_timeout_source, test_release, test_run_all_changed, test_slow_frames, test_switch_table, test_target_image_pool, test_tools_lock, test_upscaler_rows, test_vcore_escape, test_vk_pgo, test_walk_key.

Not tested: a real game started by the launcher while a timed job is queued: the tests use real Windows processes named longshot.exe, and the queue's waiting on user_playing() is unchanged. Nobody was playing on this PC during these runs.

Must be tested:

  • None

Risks

What could break: on Windows, the queue now holds timed and --when-idle jobs while a longshot.exe started outside the queue runs, which is the point. A game started by a script outside the queue looks like a player until it ends. Run it through the queue (it sets LONGSHOT_JOB), or set LONGSHOT_QUEUE_GAMES. Reading a process's memory needs only read access to the user's own processes and changes nothing in them; an elevated or another user's game isn't readable and counts as nobody's, as on Linux.

Turning it off: revert the commit

Compatibility: Nothing changes for a player. On Windows, job_queue.py status --json now fills games and builds_outside.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.
**Type:** fix · **Risk:** low · **Tested on:** Windows 11, RTX 5090 (Python 3.14); Linux (WSL Ubuntu 24.04, Python 3.12) · **Needs:** none ## TL;DR On Windows, the PC's job queue never noticed someone playing. A timed measurement or a "wait until idle" job could start while the game was running and spoil both, and `status` always said nobody was playing. The queue now spots a running game on Windows the way it already does on Linux, and also sees builds started outside it. New tests fail on main on Windows and Linux and pass here, including real processes on Windows. ## Breakdown - **Who it helps:** anyone sharing a Windows PC with agents or scripts that run through the job queue, and the measurements made there. - **What was wrong:** on Windows the queue's player check returned "nobody" without looking, and its check for builds outside the queue was skipped. - **What changes:** on Windows the queue reads the process list and each game's environment, as Linux reads `/proc`. A game not started by a queued job holds timed and `--when-idle` jobs, and `status` names it. - **What does not change:** Linux's behaviour (same results before and after), the queue's rules, slots and order, and anything in the game. - **How it was checked:** new tests over a made-up process table fail on main on both OSes and pass here; real `longshot.exe` processes on Windows are told apart by `LONGSHOT_JOB`; full suites on both OSes show no new failure. - **What is still open:** nothing. - **Risk:** low: only the queue's script, and the Windows side only reads processes. ## Why AGENTS.md and the measure skill send every game run, measurement and profile on a PC through `scripts/job_queue.py`, so that nothing runs over a person playing or over another measurement. On Windows that protection was missing. At main 310e7c9d: - `user_playing()` returns False whenever `WINDOWS` (the check before its `game_procs` call); - `status` sets `games = []` and `heavy = []` on Windows, so it always prints `playing: nobody` and never a build outside the queue; - `game_procs`, `heavy_procs`, `argv_of`, `env_of` and `tree` read `/proc` only. On Windows they find nothing, so a timed run never waits for a build outside the queue either. So a `timing`/`profile` job or a `--when-idle` ticket on Windows starts while someone plays, and its numbers are measured over the game. The PC's release-check runner found this. Prior work: none found. I searched `job_queue`, `user_playing`, `game_procs`, `heavy_procs`, "player protection", "playing: nobody", "you are playing" and the team chat. #1315 (NitnekLeBorgne) changes only the queue's history. #1085 and #1100 built the queue and its Windows job objects, and neither covers this. ## How to reproduce **Seen on:** Windows 11, RTX 5090, main 310e7c9d **Settings and switches:** defaults **Steps:** Start the game (`longshot.exe`), then run `python -I scripts/job_queue.py status`, or queue `python -I scripts/job_queue.py run fps --mode timing -- <anything>`. Game-free: `python -I -B tests/test_job_queue.py` from this PR's test commit. **Expected:** `playing: longshot.exe (pid N)`, and the timed job waits: "fps waits: you are playing" **Actual:** `playing: nobody`, and the timed job starts at once ## What changed The queue's process reads now work on Windows, so the player check, `status` and the stray-build check work there as they do on Linux. - queue: `scripts/job_queue.py`: - `_win_snapshot()` lists every process with its parent and program (Toolhelp). - `_win_params()` reads a process's command line and environment from its own PEB (`NtQueryInformationProcess`, `ReadProcessMemory`), as `/proc/<pid>/cmdline` and `environ`. It only reads a 64-bit process the user may read; anything else is unreadable, as on Linux. - `all_pids`, the new `exes`/`exe_of`/`comm_of`, `argv_of`, `env_of` (Windows' `=C:` entries are no variables) and `tree` (by parent, a child born after its parent) use these on Windows. - `game_procs` and `heavy_procs` first match the program's name, so only games' and pythons' memory is read. They also accept backslashed script paths. - `user_playing()` and `status` no longer skip Windows. - docs: `docs/development.md` ("On Windows too") and the script's usage say how a player is seen. - tests: `tests/test_job_queue.py` (first commit) adds two classes: - `WindowsPlayers` runs the Windows code over a made-up process table on any OS. It covers a player, a job's game, shader preparation, an unreadable game, builds outside the queue with backslashes, a job's tree and a reused pid. - `WindowsPlayersReal` (Windows only) uses a copy of `ping.exe` named `longshot.exe`, with and without `LONGSHOT_JOB`, and a python running `nx2/build.py`. ## Measurements **Verdict:** fixed. On main the new tests fail on Windows (7) and Linux (5, the real-process ones being Windows-only); here they all pass. Linux's own detection is unchanged. Hardware: Windows 11, RTX 5090, Python 3.14; WSL Ubuntu 24.04, Python 3.12 Scene: none: game-free checks Method: `python -I -B tests/test_job_queue.py -v` on main (with this PR's test) and on this PR, on Windows and Linux. The fix was then broken on purpose and the tests run again. On Linux, a copy of `sleep` named `longshot` was run with and without `LONGSHOT_JOB` through `game_procs`/`user_playing`, on main and here. How to reproduce: the commands above. Rebased since onto main 4b68c42b (its four commits touch only `docs/development.md` elsewhere among these files); `test_job_queue.py` gives the same result there. Rebased again onto main 1055077d (head f8b14525; main changed neither `scripts/job_queue.py` nor `tests/test_job_queue.py`): this PR | | Before: main @ 310e7c9d (with the test: 93f9a3b8) | After: this PR @ 716cccd3 (64d0afbc on 4b68c42b; f8b14525 on 1055077d) | |---|---|---| | new tests, Windows | 7 of 7 fail (1 error) | 7 of 7 pass | | new tests, Linux | 5 of 5 fail (2 skipped: Windows only) | 5 of 5 pass | | Linux real game, no job / under a job | yours / `probe`, playing: True | yours / `probe`, playing: True | | `status` on this PC, a build running beside it under pc-lock | (nothing: skipped on Windows) | `(a build outside the queue: pid 181368 ...python.exe -I nx2\make_game...)` | ``` Before: 93f9a3b8 (main 310e7c9d + the test), Windows FAIL: test_games: AssertionError: {} != {40: '', 41: 'fps', 42: 'prepare', 43: None} FAIL: test_playing: AssertionError: False is not true : a longshot.exe of nobody's job: someone is playing FAIL: test_status: 'playing: longshot.exe (pid 40)' not found in 'playing: nobody\n...' FAIL: test_builds_outside: Lists differ: [] != [50, 53] FAIL: test_tree: Lists differ: [60] != [53, 60, 61] FAIL: test_a_build_outside_the_queue: AssertionError: 509708 not found in [] ERROR: test_a_game_and_a_jobs_game Ran 19 tests FAILED (failures=7, errors=1, skipped=1) (one of the 7: test_ram_file_system, failing on main on Windows already) After: 716cccd3, Windows test_builds_outside ... ok / test_games ... ok / test_playing ... ok / test_status ... ok / test_tree ... ok test_a_build_outside_the_queue ... ok / test_a_game_and_a_jobs_game ... ok (test_ram_file_system still FAIL: as on main) After, Linux: Ran 19 tests OK (skipped=5) Broken on purpose (Windows): main's `if WINDOWS` put back in user_playing: test_playing and test_a_game_and_a_jobs_game fail; the environment's "=C:" entries kept: test_games fails ``` ### Flags and settings None: no switch is added or changed. `LONGSHOT_QUEUE_GAMES` and `LONGSHOT_QUEUE_HEAVY` now apply on Windows too, with their defaults (`longshot.exe` is already in the games list). ## Testing **Tested:** - `tests/test_job_queue.py` on my own PC (Windows 11), on main and here, and in WSL Ubuntu 24.04, on main and here (table above). The real-process tests ran on Windows: a `longshot.exe` without `LONGSHOT_JOB` is a player, one with `LONGSHOT_JOB=probe` is that job's, its arguments are read, and a python running `nx2/build.py` is a build outside the queue unless it's in the caller's tree. - `python -I scripts/job_queue.py status` on this PC: `playing: nobody` (no game running) in 0.39 s, and it named the release tester's `make_game.py` running beside it as a build outside the queue. - `python -I -B tests/run_all.py`, this PR @ 64d0afbc and main @ 4b68c42b in a clean checkout, the same runs. **Windows 11** (12 jobs): this PR 665 passed, 50 failed; main 664 passed, 51 failed. `test_longshot_ctl` failed only on main, a timeout it sometimes has under load, not this PR's. **Linux, WSL Ubuntu 24.04** (8 jobs): 594 passed, 50 failed on both. None fails here and not on main. `test_job_queue` fails on Windows on both, for its `test_ram_file_system` alone (its new cases pass here), and passes on Linux on both. The 50 that fail on both, on this PC: Windows `test_auto_handheld`, `test_bug_report_defaults`, `test_build_long_paths`, `test_compute_warm_apart`, `test_cpu_report`, `test_crash_dialog`, `test_crt_options`, `test_damage_debug_menu`, `test_dev_packs`, `test_discover`, `test_display_startup_failure`, `test_dlss_feature_size`, `test_draw_stream_visibility`, `test_failed_compile_keeps_objects`, `test_fsr_image_retirement`, `test_fsr_uniform_ownership`, `test_guest_imported`, `test_hud_layout_row`, `test_import_reach`, `test_invalidate_quick`, `test_invalidate_vcore`, `test_job_queue`, `test_libc_hooks`, `test_lift_lookups`, `test_linux_preflight`, `test_linux_pressure`, `test_mirror_budget`, `test_missing_nup`, `test_pads_after_menu`, `test_parallel_init_order`, `test_pending_guest_import`, `test_pipeline_play_cache`, `test_present_thread`, `test_profiling_mode`, `test_psychov35_fsr3_fg`, `test_readback_timeout_source`, `test_recorder_sync`, `test_replay_write_order`, `test_run_all_changed`, `test_sdiv_overflow`, `test_slow_frames`, `test_staging_burst`, `test_switch_table`, `test_target_image_pool`, `test_tested_configurations`, `test_upscaler_rows`, `test_vcore_escape`, `test_vk_occlusion_queries`, `test_walk`, `test_walk_key`; Linux `byte_compile_tracked`, `test_bake_ds`, `test_bug_report`, `test_bug_report_defaults`, `test_check_third_party`, `test_control`, `test_cpu_report`, `test_crash_groups`, `test_crt_options`, `test_crt_shaders`, `test_damage_debug_menu`, `test_dev_packs`, `test_display_startup_failure`, `test_dlss_feature_size`, `test_failed_compile_keeps_objects`, `test_finish_output`, `test_guest_imported`, `test_hardware_report`, `test_hud_layout_row`, `test_invalidate_quick`, `test_invalidate_vcore`, `test_libc_hooks`, `test_link_skip`, `test_linux_downgrade`, `test_linux_pressure`, `test_linux_update`, `test_log_open_failure`, `test_mirror_budget`, `test_missing_nup`, `test_mods_size_table`, `test_oot_state_tap`, `test_pending_guest_import`, `test_pipeline_cache_file`, `test_pipeline_cache_save`, `test_pipeline_play_cache`, `test_pipeline_tour`, `test_present_thread`, `test_profiling_mode`, `test_psychov35_fsr3_fg`, `test_readback_timeout_source`, `test_release`, `test_run_all_changed`, `test_slow_frames`, `test_switch_table`, `test_target_image_pool`, `test_tools_lock`, `test_upscaler_rows`, `test_vcore_escape`, `test_vk_pgo`, `test_walk_key`. **Not tested:** a real game started by the launcher while a timed job is queued: the tests use real Windows processes named `longshot.exe`, and the queue's waiting on `user_playing()` is unchanged. Nobody was playing on this PC during these runs. **Must be tested:** - None ## Risks **What could break:** on Windows, the queue now holds timed and `--when-idle` jobs while a `longshot.exe` started outside the queue runs, which is the point. A game started by a script outside the queue looks like a player until it ends. Run it through the queue (it sets `LONGSHOT_JOB`), or set `LONGSHOT_QUEUE_GAMES`. Reading a process's memory needs only read access to the user's own processes and changes nothing in them; an elevated or another user's game isn't readable and counts as nobody's, as on Linux. **Turning it off:** revert the commit **Compatibility:** Nothing changes for a player. On Windows, `job_queue.py status --json` now fills `games` and `builds_outside`. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated.
dukenheimer force-pushed fix/job-queue-windows-player from 716cccd3f0
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 64d0afbc6e
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-18 01:56:15 +00:00
Compare
Author
Contributor

Rebased onto main 4b68c42b (head 64d0afbc); test_job_queue.py gives the same result there. The full run_all.py on Windows and Linux, on this PR and on main, follows after the hat agent's build-slot turn; then I mark it ready.

Rebased onto main 4b68c42b (head 64d0afbc); test_job_queue.py gives the same result there. The full run_all.py on Windows and Linux, on this PR and on main, follows after the hat agent's build-slot turn; then I mark it ready.
Author
Contributor

Full python -I -B tests/run_all.py, this PR @ 64d0afbc against main @ 4b68c42b (a clean checkout), the same runs:

  • Windows 11, 12 jobs: this PR 665 passed, 50 failed; main 664 passed, 51 failed. Only main fails test_longshot_ctl, a timeout it sometimes has under load, not this PR's doing.
  • Linux, WSL Ubuntu 24.04, 8 jobs: 594 passed, 50 failed, on both.
  • New failures: none on either OS.
  • test_job_queue: on Windows it fails on both, for its test_ram_file_system alone; its new cases pass here. On Linux it passes on both.

The description lists the 50 checks that fail on both. Marking it ready.

Full `python -I -B tests/run_all.py`, this PR @ 64d0afbc against main @ 4b68c42b (a clean checkout), the same runs: - **Windows 11, 12 jobs:** this PR 665 passed, 50 failed; main 664 passed, 51 failed. Only main fails `test_longshot_ctl`, a timeout it sometimes has under load, not this PR's doing. - **Linux, WSL Ubuntu 24.04, 8 jobs:** 594 passed, 50 failed, on both. - **New failures:** none on either OS. - **`test_job_queue`:** on Windows it fails on both, for its `test_ram_file_system` alone; its new cases pass here. On Linux it passes on both. The description lists the 50 checks that fail on both. Marking it ready.
dukenheimer changed title from WIP: fix(queue): on Windows the job queue waits for a player and sees builds outside it, as on Linux to fix(queue): on Windows the job queue waits for a player and sees builds outside it, as on Linux 2026-11-18 02:37:10 +00:00
dukenheimer force-pushed fix/job-queue-windows-player from 64d0afbc6e
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to f8b145252e
Some checks are pending
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
2026-11-19 01:50:02 +00:00
Compare
HumanBean left a comment

From the code and the evidence, this looks ready.

I read the Windows process reads against their structures. PROCESSENTRY32W, PROCESS_BASIC_INFORMATION (six pointers, the PEB at [1]), PEB.ProcessParameters at 0x20, and RTL_USER_PROCESS_PARAMETERS (CommandLine at 0x70/0x78, Environment at 0x80, EnvironmentSize at 0x3F0) are all right for x64. The read refuses WOW64 targets and a 32-bit Python. Nothing new is installed (ctypes only) and no new command runs: CommandLineToArgvW only parses, and the command line it parses is never executed. The PR asks only for PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_VM_READ. That fails for another user's process or an elevated one, so such a game reads as job: None, which never counts as a player, the same as on Linux. Only processes whose program is in LONGSHOT_QUEUE_GAMES, and pythons, have their memory read. The Builder (LongshotBuilder.exe) and the launcher are not in the games list, so they are never mistaken for a player. A build's longshot.exe --prepare-shaders still counts as prepare. On Linux, game_procs, heavy_procs and tree read the same /proc files as before. The only changes there are sorted pid order, a case-insensitive python match and backslash-tolerant script paths, and none of these changes a result in practice.

Suggestion (not blocking): hookshot and lichii are never seen on Windows. In scripts/job_queue.py:592, the Windows pre-filter compares the program's full file name with the list. The default list is longshot,longshot.exe,hookshot,lichii, so on Windows only longshot.exe can match: hookshot.exe or lichii.exe running there doesn't hold a timed run, while on Linux they do. The same goes for a LONGSHOT_QUEUE_GAMES=longshot someone sets. Also accepting the name without .exe would fix both, for example n = exe.lower(); if WINDOWS and n not in names and n.removesuffix(".exe") not in names: continue. The hit lookup below it would need the same.

Measurements: this is a tooling fix with no game scene, so the evidence is the tests. The new WindowsPlayers cases run the Windows path over a made-up process table on any OS and would fail without the fix: on main, game_procs and tree read /proc, and user_playing returns early on Windows. The author reports 7 of 7 failing on main and passing here on Windows 11, and 5 of 5 on Linux. They also broke the fix on purpose and showed the right cases fail. WindowsPlayersReal exercises the real Toolhelp and PEB reads on Windows (a copied ping.exe named longshot.exe, with and without LONGSHOT_JOB). That covers what this changes. A real game started from the launcher while a timed job waits wasn't run, which the PR says. It would be a nice confirmation but isn't needed to merge.

From the code and the evidence, this looks ready. I read the Windows process reads against their structures. `PROCESSENTRY32W`, `PROCESS_BASIC_INFORMATION` (six pointers, the PEB at [1]), `PEB.ProcessParameters` at 0x20, and `RTL_USER_PROCESS_PARAMETERS` (`CommandLine` at 0x70/0x78, `Environment` at 0x80, `EnvironmentSize` at 0x3F0) are all right for x64. The read refuses WOW64 targets and a 32-bit Python. Nothing new is installed (ctypes only) and no new command runs: `CommandLineToArgvW` only parses, and the command line it parses is never executed. The PR asks only for `PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_VM_READ`. That fails for another user's process or an elevated one, so such a game reads as `job: None`, which never counts as a player, the same as on Linux. Only processes whose program is in `LONGSHOT_QUEUE_GAMES`, and pythons, have their memory read. The Builder (`LongshotBuilder.exe`) and the launcher are not in the games list, so they are never mistaken for a player. A build's `longshot.exe --prepare-shaders` still counts as `prepare`. On Linux, `game_procs`, `heavy_procs` and `tree` read the same `/proc` files as before. The only changes there are sorted pid order, a case-insensitive `python` match and backslash-tolerant script paths, and none of these changes a result in practice. **Suggestion (not blocking): `hookshot` and `lichii` are never seen on Windows.** In `scripts/job_queue.py:592`, the Windows pre-filter compares the program's full file name with the list. The default list is `longshot,longshot.exe,hookshot,lichii`, so on Windows only `longshot.exe` can match: `hookshot.exe` or `lichii.exe` running there doesn't hold a timed run, while on Linux they do. The same goes for a `LONGSHOT_QUEUE_GAMES=longshot` someone sets. Also accepting the name without `.exe` would fix both, for example `n = exe.lower(); if WINDOWS and n not in names and n.removesuffix(".exe") not in names: continue`. The `hit` lookup below it would need the same. Measurements: this is a tooling fix with no game scene, so the evidence is the tests. The new `WindowsPlayers` cases run the Windows path over a made-up process table on any OS and would fail without the fix: on main, `game_procs` and `tree` read `/proc`, and `user_playing` returns early on Windows. The author reports 7 of 7 failing on main and passing here on Windows 11, and 5 of 5 on Linux. They also broke the fix on purpose and showed the right cases fail. `WindowsPlayersReal` exercises the real Toolhelp and PEB reads on Windows (a copied `ping.exe` named `longshot.exe`, with and without `LONGSHOT_JOB`). That covers what this changes. A real game started from the launcher while a timed job waits wasn't run, which the PR says. It would be a nice confirmation but isn't needed to merge.
@ -418,2 +591,2 @@
except OSError:
exe = ""
for pid, exe in sorted(exes().items()):
if WINDOWS and exe.lower() not in names: # (Windows: the program's name; reading every process's memory costs)
Owner

On Windows this compares the full file name, so the default list's hookshot and lichii (and a LONGSHOT_QUEUE_GAMES=longshot someone sets) never match hookshot.exe or lichii.exe. Also accepting exe.lower().removesuffix(".exe") would keep the Windows side the same as Linux.

On Windows this compares the full file name, so the default list's `hookshot` and `lichii` (and a `LONGSHOT_QUEUE_GAMES=longshot` someone sets) never match `hookshot.exe` or `lichii.exe`. Also accepting `exe.lower().removesuffix(".exe")` would keep the Windows side the same as Linux.
@ -452,3 +622,2 @@
out = []
for pid in all_pids():
if pid in skip:
for pid, exe in sorted(exes().items()):
Owner

Nit: on Linux, exes() now readlinks /proc/<pid>/exe for every process on each call (every 2 s in _keep_apart while a timed run measures), and heavy_procs never uses the result there. all_pids() off Windows would save that, e.g. for pid, exe in (sorted(exes().items()) if WINDOWS else ((p, "") for p in all_pids())):.

Nit: on Linux, `exes()` now readlinks `/proc/<pid>/exe` for every process on each call (every 2 s in `_keep_apart` while a timed run measures), and `heavy_procs` never uses the result there. `all_pids()` off Windows would save that, e.g. `for pid, exe in (sorted(exes().items()) if WINDOWS else ((p, "") for p in all_pids())):`.
HumanBean deleted branch fix/job-queue-windows-player 2026-11-19 07:35:05 +00:00
Some checks are pending
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
Some workflows are waiting to be reviewed.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1523
No description provided.