fix(queue): preserve complete history around malformed lines #1315

Open
NitnekLeBorgne wants to merge 2 commits from NitnekLeBorgne/Longshot-contributions:fix/job-history-partial-lines-20261117 into main
Contributor

Type: fix · Risk: low · Tested on: Linux and native Windows, game-free contributor-tool fixtures; no GPU/driver exercised · Needs: none

Why

An interrupted history append can hide previously complete jobs from the history command. This fault-injection reproduction concerns recovery of a JSONL history file, not a claimed observed user crash.

The issue tracker was searched again on 2026-11-17 using jsonl/job history; no matching issue was found in the returned, observational corpus. This is not a claim about hidden/deleted issues; no issue closure is claimed.

How to reproduce

Seen on: Linux and native Windows, offline fixtures; GPU and driver not involved.
Settings and switches: Private synthetic fixtures; no game settings or GAME_PATHS inputs.
Steps: Use tests/test_job_queue_history.py: write three complete producer-format records followed by a prefix of a fourth, then invoke only the history CLI with --json on the private fixture. Run python -I -B -m unittest discover -s tests -p test_job_queue_history.py against baseline production plus the added tests, then the combined candidate.
Expected: Complete records remain in append order and malformed lines are reported once on stderr; stdout remains valid JSON.
Actual: The baseline CLI exits 0 with [] and no warning, losing the complete records from the displayed history.

What changed

Decode nonblank history lines individually, keep complete records in order, then apply the existing [-n:] slice. Emit one aggregated malformed-line diagnostic without repairing or rewriting the file. The patch is limited to the following production/test pair, both Git mode 100644.

  • Production: scripts/job_queue.py — history() only.
  • Regression coverage: tests/test_job_queue_history.py — supported failure and neighboring controls.

Rebased onto main 544229a260c5f5008f241b9cf48ae6b01c1602b9. Test-only commit: 8ad9b36895aec1f7c9d310b437a78a2f7a22eac6. Final fix head: c92f4a526518709e76a243b98315632baacf87f6. The changed file bytes/modes remain identical to the independently reviewed version; complete current-base trees and parent chains were verified. Fresh normal qualification uses this actual HEAD.

Measurements

Nothing to measure: this is a contributor-tool correctness fix, not a gameplay, rendering or performance change.

Flags and settings

None

Testing

Tested: Linux: 8 history methods PASS with 14 real CLI calls. Independent SPEC additionally checks 15 candidate CLI cases; its total 30 CLI invocations includes RED and author-suite calls, not 30 unique tests. Native Windows: 8 history methods PASS; native exit 0. Public scripts ran with python -I -B and owned fixtures. Native receipts: Job final 0, complete stdout/stderr, no controller errors; all 818 source paths unchanged. V5's three native controls and 60-child burst passed independent verification. Validated short-lived-process metadata gaps remain: no universal visibility or filesystem/network sandbox claim. The retained native receipt is from original head b7166c7945e8add220e682bd26555713e64cdf20; the production/test bytes, modes and native dependency closure used there are unchanged by this rebase. No new Windows run is claimed.

Fresh normal Linux qualification on the pinned current base and actual rebased candidate used python -I -B tests/run_all.py --jobs 2 --junit ..., identical interpreter/environment policy and real Git HEAD/index:

  • Main @ 544229a260c5f5008f241b9cf48ae6b01c1602b9: 721 checks, 571 PASS / 42 FAIL / 108 SKIP, native exit 1.
  • This PR @ c92f4a526518709e76a243b98315632baacf87f6: 722 checks, 572 PASS / 42 FAIL / 108 SKIP, native exit 1.
  • The added passing check is tests/test_job_queue_history.py. All 721 common statuses, skip reasons and complete failure payloads agree after narrowly recorded diagnostic-noise handling: owned source/run paths, source-proven temporary file/directory suffixes, unittest elapsed times, one GCC intermediate object and one synthetic-fixture backup directory. Assertion values, failure identities, symbols, numeric offsets and statuses are not normalized. The GCC/linker error still names choose_face, hos_options_file, .text+0xa1a and exit 1; a separate retained compiler/assembler/linker trace proves this toolchain's temporary-object provenance. No universal no-regressions claim.
  • Tracked source bytes/modes remained unchanged; each owned process group was absent at completion. Exports omit the protected repository instruction file: this is source-only qualification, not literal-checkout or hosted-CI equivalence. The earlier source-only baseline lacking Git metadata and the historical night's strict-false comparison are retained separately, not relabelled as passing results.

The 42 normal failures also observed on this current-main qualification are named here, as the template permits; the normal suite is not green:

  • tests/test_bug_report.py
  • tests/test_bug_report_defaults.py
  • tests/test_cpu_report.py
  • tests/test_crash_groups.py
  • tests/test_crt_options.py
  • tests/test_damage_debug_menu.py
  • tests/test_discover.py
  • tests/test_display_startup_failure.py
  • tests/test_dlss_feature_size.py
  • tests/test_failed_compile_keeps_objects.py
  • tests/test_framegen_preset.py
  • tests/test_guest_imported.py
  • tests/test_hardware_report.py
  • tests/test_hud_layout_row.py
  • tests/test_invalidate_vcore.py
  • tests/test_lift_lookups.py
  • tests/test_link_skip.py
  • tests/test_linux_pressure.py
  • tests/test_linux_update.py
  • tests/test_longshot_saves.py
  • tests/test_mirror_budget.py
  • tests/test_missing_nup.py
  • tests/test_mods_size_table.py
  • tests/test_pending_guest_import.py
  • tests/test_pipeline_play_cache.py
  • tests/test_present_thread.py
  • tests/test_profiling_mode.py
  • tests/test_readback_timeout_source.py
  • tests/test_renderer_new_handler.py
  • tests/test_rumble_menu.py
  • tests/test_run_all_changed.py
  • tests/test_slow_frames.py
  • tests/test_switch_table.py
  • tests/test_target_image_pool.py
  • tests/test_tested_configurations.py
  • tests/test_upscaler_round_trip.py
  • tests/test_upscaler_rows.py
  • tests/test_vcore_escape.py
  • tests/test_vk_occlusion_queries.py
  • tests/test_vk_swapchain_compat.py
  • tests/test_walk.py
  • tests/test_walk_key.py

Not tested: Full Windows suite, real SDK/release installation, gameplay/GPU or hosted-CI equivalence. These source/fixture-only limits do not invent extra native-test obligations. Hosted checks are tracked on the final exact public head; readiness is not approval, a full CI PASS or merge eligibility.

Must be tested: None additional for this two-file contributor-tool change. The normal-runner exception is explicitly limited to the named current-main failures above, with source-only context disclosed; it is not a waiver for a new candidate failure or an unknown comparison.

Risks

What could break: History parsing/diagnostics only. Invalid UTF-8 and syntactically valid out-of-schema JSON are outside the new contract; producer, dispatch, execution, cancellation and existing slice semantics are unchanged.

Turning it off: Revert the fix and its associated test commit.

Compatibility: No game runtime, saves, options, shader/pipeline cache or vk ABI changes; only the stated contributor-tool behavior changes. No GAME_PATHS directory is touched and no in-game run is claimed.

Checklist

The normal runner remains native exit 1; its named current-main failures use the template's explicit exception, not a full-suite PASS claim. Tracker search and the GPL-3.0-or-later diff/provenance check are complete. No incompatible dependency, third-party code, dump or decompiled/generated game content is introduced.

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.

The retained behavioral RED uses identical baseline production and regression-test bytes; the rebased normal run verifies the final source on the current-base context. No host C/C++, shader, game switch or visible-game change applies.

**Type:** fix · **Risk:** low · **Tested on:** Linux and native Windows, game-free contributor-tool fixtures; no GPU/driver exercised · **Needs:** none ## Why An interrupted history append can hide previously complete jobs from the history command. This fault-injection reproduction concerns recovery of a JSONL history file, not a claimed observed user crash. The issue tracker was searched again on 2026-11-17 using jsonl/job history; no matching issue was found in the returned, observational corpus. This is not a claim about hidden/deleted issues; no issue closure is claimed. ## How to reproduce **Seen on:** Linux and native Windows, offline fixtures; GPU and driver not involved. **Settings and switches:** Private synthetic fixtures; no game settings or GAME_PATHS inputs. **Steps:** Use tests/test_job_queue_history.py: write three complete producer-format records followed by a prefix of a fourth, then invoke only the history CLI with --json on the private fixture. Run `python -I -B -m unittest discover -s tests -p test_job_queue_history.py` against baseline production plus the added tests, then the combined candidate. **Expected:** Complete records remain in append order and malformed lines are reported once on stderr; stdout remains valid JSON. **Actual:** The baseline CLI exits 0 with [] and no warning, losing the complete records from the displayed history. ## What changed Decode nonblank history lines individually, keep complete records in order, then apply the existing [-n:] slice. Emit one aggregated malformed-line diagnostic without repairing or rewriting the file. The patch is limited to the following production/test pair, both Git mode `100644`. - Production: `scripts/job_queue.py` — history() only. - Regression coverage: `tests/test_job_queue_history.py` — supported failure and neighboring controls. Rebased onto main `544229a260c5f5008f241b9cf48ae6b01c1602b9`. Test-only commit: `8ad9b36895aec1f7c9d310b437a78a2f7a22eac6`. Final fix head: `c92f4a526518709e76a243b98315632baacf87f6`. The changed file bytes/modes remain identical to the independently reviewed version; complete current-base trees and parent chains were verified. Fresh normal qualification uses this actual HEAD. ## Measurements Nothing to measure: this is a contributor-tool correctness fix, not a gameplay, rendering or performance change. ### Flags and settings None ## Testing **Tested:** Linux: 8 history methods PASS with 14 real CLI calls. Independent SPEC additionally checks 15 candidate CLI cases; its total 30 CLI invocations includes RED and author-suite calls, not 30 unique tests. Native Windows: 8 history methods PASS; native exit 0. Public scripts ran with `python -I -B` and owned fixtures. Native receipts: Job final 0, complete stdout/stderr, no controller errors; all 818 source paths unchanged. V5's three native controls and 60-child burst passed independent verification. Validated short-lived-process metadata gaps remain: no universal visibility or filesystem/network sandbox claim. The retained native receipt is from original head `b7166c7945e8add220e682bd26555713e64cdf20`; the production/test bytes, modes and native dependency closure used there are unchanged by this rebase. No new Windows run is claimed. Fresh normal Linux qualification on the pinned current base and actual rebased candidate used `python -I -B tests/run_all.py --jobs 2 --junit ...`, identical interpreter/environment policy and real Git HEAD/index: - Main @ `544229a260c5f5008f241b9cf48ae6b01c1602b9`: 721 checks, 571 PASS / 42 FAIL / 108 SKIP, native exit 1. - This PR @ `c92f4a526518709e76a243b98315632baacf87f6`: 722 checks, 572 PASS / 42 FAIL / 108 SKIP, native exit 1. - The added passing check is `tests/test_job_queue_history.py`. All 721 common statuses, skip reasons and complete failure payloads agree after narrowly recorded diagnostic-noise handling: owned source/run paths, source-proven temporary file/directory suffixes, unittest elapsed times, one GCC intermediate object and one synthetic-fixture backup directory. Assertion values, failure identities, symbols, numeric offsets and statuses are not normalized. The GCC/linker error still names `choose_face`, `hos_options_file`, `.text+0xa1a` and exit 1; a separate retained compiler/assembler/linker trace proves this toolchain's temporary-object provenance. No universal no-regressions claim. - Tracked source bytes/modes remained unchanged; each owned process group was absent at completion. Exports omit the protected repository instruction file: this is source-only qualification, not literal-checkout or hosted-CI equivalence. The earlier source-only baseline lacking Git metadata and the historical night's strict-false comparison are retained separately, not relabelled as passing results. The 42 normal failures also observed on this current-main qualification are named here, as the template permits; the normal suite is **not green**: - `tests/test_bug_report.py` - `tests/test_bug_report_defaults.py` - `tests/test_cpu_report.py` - `tests/test_crash_groups.py` - `tests/test_crt_options.py` - `tests/test_damage_debug_menu.py` - `tests/test_discover.py` - `tests/test_display_startup_failure.py` - `tests/test_dlss_feature_size.py` - `tests/test_failed_compile_keeps_objects.py` - `tests/test_framegen_preset.py` - `tests/test_guest_imported.py` - `tests/test_hardware_report.py` - `tests/test_hud_layout_row.py` - `tests/test_invalidate_vcore.py` - `tests/test_lift_lookups.py` - `tests/test_link_skip.py` - `tests/test_linux_pressure.py` - `tests/test_linux_update.py` - `tests/test_longshot_saves.py` - `tests/test_mirror_budget.py` - `tests/test_missing_nup.py` - `tests/test_mods_size_table.py` - `tests/test_pending_guest_import.py` - `tests/test_pipeline_play_cache.py` - `tests/test_present_thread.py` - `tests/test_profiling_mode.py` - `tests/test_readback_timeout_source.py` - `tests/test_renderer_new_handler.py` - `tests/test_rumble_menu.py` - `tests/test_run_all_changed.py` - `tests/test_slow_frames.py` - `tests/test_switch_table.py` - `tests/test_target_image_pool.py` - `tests/test_tested_configurations.py` - `tests/test_upscaler_round_trip.py` - `tests/test_upscaler_rows.py` - `tests/test_vcore_escape.py` - `tests/test_vk_occlusion_queries.py` - `tests/test_vk_swapchain_compat.py` - `tests/test_walk.py` - `tests/test_walk_key.py` **Not tested:** Full Windows suite, real SDK/release installation, gameplay/GPU or hosted-CI equivalence. These source/fixture-only limits do not invent extra native-test obligations. Hosted checks are tracked on the final exact public head; readiness is not approval, a full CI PASS or merge eligibility. **Must be tested:** None additional for this two-file contributor-tool change. The normal-runner exception is explicitly limited to the named current-main failures above, with source-only context disclosed; it is not a waiver for a new candidate failure or an unknown comparison. ## Risks **What could break:** History parsing/diagnostics only. Invalid UTF-8 and syntactically valid out-of-schema JSON are outside the new contract; producer, dispatch, execution, cancellation and existing slice semantics are unchanged. **Turning it off:** Revert the fix and its associated test commit. **Compatibility:** No game runtime, saves, options, shader/pipeline cache or vk ABI changes; only the stated contributor-tool behavior changes. No GAME_PATHS directory is touched and no in-game run is claimed. ## Checklist The normal runner remains native exit 1; its named current-main failures use the template's explicit exception, not a full-suite PASS claim. Tracker search and the GPL-3.0-or-later diff/provenance check are complete. No incompatible dependency, third-party code, dump or decompiled/generated game content is introduced. ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. The retained behavioral RED uses identical baseline production and regression-test bytes; the rebased normal run verifies the final source on the current-base context. No host C/C++, shader, game switch or visible-game change applies.
Assisted-by: Hermes Agent (OpenAI GPT-6.1-sol)
Co-Authored-By: Claudia (OpenAI GPT-6.1-sol)
fix(queue): preserve complete history around malformed lines
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
b7166c7945
Assisted-by: Hermes Agent (OpenAI GPT-6.1-sol)
Co-Authored-By: Claudia (OpenAI GPT-6.1-sol)
NitnekLeBorgne force-pushed fix/job-history-partial-lines-20261117 from b7166c7945
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to c92f4a5265
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 8s
2026-11-17 12:26:11 +00:00
Compare
Author
Contributor

Rebased onto main 544229a260c5f5008f241b9cf48ae6b01c1602b9 with both reviewed file bytes and modes preserved. New head: c92f4a526518709e76a243b98315632baacf87f6. Fresh source-only Linux normal qualification is 572 PASS / 42 named FAIL / 108 SKIP versus current main 571 / 42 / 108; both native exit 1. The new regression module passes; common failure/skip diagnostics match after narrowly documented temporary-path and timing handling, without hiding assertion numbers or linker symbols. The description now names the current-main failures and keeps the qualification limits explicit. The retained native Windows 8-method proof covers the unchanged source closure; no new Windows, gameplay or full-CI result is claimed.

Rebased onto main `544229a260c5f5008f241b9cf48ae6b01c1602b9` with both reviewed file bytes and modes preserved. New head: `c92f4a526518709e76a243b98315632baacf87f6`. Fresh source-only Linux normal qualification is 572 PASS / 42 named FAIL / 108 SKIP versus current main 571 / 42 / 108; both native exit 1. The new regression module passes; common failure/skip diagnostics match after narrowly documented temporary-path and timing handling, without hiding assertion numbers or linker symbols. The description now names the current-main failures and keeps the qualification limits explicit. The retained native Windows 8-method proof covers the unchanged source closure; no new Windows, gameplay or full-CI result is claimed.
NitnekLeBorgne changed title from WIP: fix(queue): preserve complete history around malformed lines to fix(queue): preserve complete history around malformed lines 2026-11-17 12:26:20 +00:00
Owner

Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).

  • Tested on my own PC: ticked: Testing's Tested: the author's Linux run of the eight history methods and a native Windows run of the same methods.
  • Nothing gets worse that I know of: ticked: Measurements say nothing to measure for this contributor-tool fix, and Risks names no speed, latency or memory cost.
Brought up to the [ready-for-review standard](https://git.magicbean.dev/MagicBeanDev/Longshot/src/branch/main/CONTRIBUTING.md#ready-for-review-out-of-wip) (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know). - *Tested on my own PC*: ticked: Testing's Tested: the author's Linux run of the eight history methods and a native Windows run of the same methods. - *Nothing gets worse that I know of*: ticked: Measurements say nothing to measure for this contributor-tool fix, and Risks names no speed, latency or memory cost.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 8s
This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/job-history-partial-lines-20261117:NitnekLeBorgne-fix/job-history-partial-lines-20261117'
git switch 'NitnekLeBorgne-fix/job-history-partial-lines-20261117'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1315
No description provided.