MagicBeanDev/Longshot
MagicBeanDev/Longshot
Watch
3
0
Fork
You've already forked Longshot
 10
Code Issues 25 Pull requests 48 Projects 1 Releases Packages Wiki Activity Actions 85

WIP: fix(docs): give tested runs independent result files #96

Draft
mooki wants to merge 1 commit from mooki/Longshot:fix/tested-rows into main
pull from: mooki/Longshot:fix/tested-rows
merge into: MagicBeanDev:main
Conversation 0 Commits 1 Files changed 91 +629 -97
mooki commented 2026-11-13 02:36:25 +00:00
First-time contributor

Why

Independent test results should not force unrelated pull requests to resolve the same table insertion point. This keeps the evidence requirement while giving every measured run its own file. See #45.

What changed

  • docs/tested/: one three-line Markdown file per run. Each new row is self-contained, with a unique date/topic/tester/run filename.
  • docs/tested/legacy/: all 82 existing rows moved without changing their text or order. The three pre-existing duplicate pairs remain visible, and as above keeps its original meaning within the archive.
  • scripts/tested_configurations.py: validates the headers, eight non-empty cells, dates, duplicates and common relative references; assembles a reading view without committing a shared generated table.
  • tests/test_tested_configurations.py: migration digest, malformed/duplicate/interleaved rows, deterministic ordering, escaped pipes, stale-view rejection and real bare-Git merge regressions. Two edits to the same existing row still conflict.
  • Contributor instructions, the PR template and its example now name the same fragment destination. The description checker itself is unchanged: it reads the description, not the evidence table.

Flags and settings

Flag What it does Default now Recommendation Helps on Doesn't help on Untested on
scripts/tested_configurations.py --output FILE New: Writes the assembled Markdown view as UTF-8 Without an option, prints to stdout Off by default: choose where to save the view Reading all results together Runtime behavior Windows (should help on: Windows)
scripts/tested_configurations.py --check FILE New: Fails if the generated view is missing or stale; writes nothing Not selected Off by default: only meaningful for an existing view Checking a saved view Runtime behavior Windows (should help on: Windows)

Both are documented in docs/tested-configurations.md and the development guide. No game setting changes.

Measurements

Base: 77f605a25f
Head: 06d7c45637
Base suite: 509 passed, 0 failed, 19 skipped.
Head suite: 510 passed, 0 failed, 19 skipped.

main: 77f605a25f (2026-11-13 UTC)
this PR: move/fix/tested-rows @ 06d7c45637
Hardware: Linux 7.2.15-1-omarchy, Git 2.56.0, Python 3.14.8; no GPU work
Scene: none; documentation and Git merge behavior
Method: pinned heads from the Forgejo API; one deterministic merge per head/design, then every pair of the same row additions; source-test output, not elapsed-time or game-performance measurements
How to reproduce: python -I -B tests/test_tested_configurations.py; python -I -B tests/test_check_pr_description.py; python -I -B tests/run_all.py --jobs 6 --strict. For a pinned branch, find its ancestor with git merge-base BASE HEAD, then run git merge-tree --write-tree -z --name-only --no-messages --merge-base=ANCESTOR BASE HEAD in a bare repository. Repeat after moving only its inserted rows to unique fragment files and restoring that branch's old table to its ancestor's text.

main @ 77f605a25 this PR @ 06d7c456
Results-table conflicts, 18 pinned heads 13 0 after row conversion
Any conflict, same 18 heads 14 2, both unrelated to results
Concurrent result-only additions, all 153 branch pairs on the same base 152 conflicting pairs 0 conflicting pairs
Existing rows retained in original order 82 82, row-byte digest unchanged
Whole game-free suite, once per commit 509 passed, 0 failed, 19 skipped 510 passed, 0 failed, 19 skipped 
Pinned-head replay: table conflicts 13 -> 0; any conflicts 14 -> 2.
Same-base pair replay: table 152/153; dated headings 137/153; fragments 0/153.
New named test: Ran 14 tests ... OK
Description checker tests: Ran 18 tests ... OK
Whole suite: 510 passed, 0 failed, 19 skipped.

The 16 moved PRs alone are 13 conflicting heads before conversion and 1 afterwards. The two additional PRs are #75 and #78. This is a newer snapshot than the issue comment's historical 14-of-16 result; it does not replace that observation.

PR Pinned head Conflicting files before Conflicting files after row conversion
#47 23cc17be6ac3 1 0
#49 440fef872d5e 1 0
#50 1652ef660014 1 0
#51 d5b2ff117905 1 0
#53 4ba4554baa59 1 0
#54 15813ff4a5ee 1 0
#55 a3fdd78b7049 0 0
#56 0ce651b604ef 1 0
#57 58143b9df475 0 0
#58 365b02c32d0f 1 0
#60 e8a9913748b3 1 0
#61 74f8248ef589 0 0
#66 629766eb2063 1 0
#67 3969145a3fd5 1 0
#68 cca2aebb03ef 2 1
#71 e673c7dddb14 1 0
#75 a4bd549f215a 2 2
#78 f3281c7c7d1b 0 0

The remaining files are docs/development.md in #68, and the removed host/hos_update.c and tests/test_linux_update_skip.py in #75. Every non-table branch change was retained. The final replay uses this PR's actual commit, not just a layout sketch.

Conversion is necessary. Unchanged remote heads do not become mergeable just because this PR merges: all 18 have a one-time layout conflict against the migrated table. Twelve sets of moved rows already pass the new validator. #50, #51, #53, #57, #67 and #68 must spell out their as above details. No remote branch was modified in these experiments.

Why not union? Forgejo 17.0.1's conflict check runs the command above in the bare base repository. Adding the committed merge=union rule left all 13 table conflicts. Reproducing its sparse merge preparation also conflicted with merge, squash and rebase when the rule was already shared by both branches. The common non-mergeable status is refused before the merge style runs.

A full local checkout did honour union. It also returned success while retaining two edits of one row, a duplicate overlapping row and two incompatible headers. Swapping merge direction changed the date order. These are local command reproductions, not a new live server test; the earlier live refusal is already recorded in the issue. The server's Git version and private configuration were not inspected.

Testing

Tested: on Linux, the 14-case fragment test, the existing 18-case description-checker test and the whole game-free suite on the exact head above. The migration regression fails against main because the fragments are absent; the merge regression separately reproduces a shared-table conflict and proves distinct files merge in a bare repository. The exact Forgejo merge commands, real-head replay and same-base pair replay are described above. The base suite has no failures; its removed updater test is not carried as a supposed known failure.

Not tested: Windows execution of the new Python file/path handling; a new live Forgejo merge of fragment PRs; game runs, game builds or frame rates. No game behavior is changed. The static Git experiments are not a measurement of CI time saved.

Needs other testers:

  • Windows: run python -I -B tests/test_tested_configurations.py, then generate and check a view with a non-ASCII output filename, expecting the tests to pass and the saved view to match (request to open)
  • Forgejo 17: merge two disposable PRs adding distinct same-date result files, expecting the second to stay mergeable after the first; confirm the new contributor workflow is wanted before doing this (request to open)

Screenshots

Nothing on screen in the game. Each source fragment is an ordinary Markdown table; the combined view is generated on demand.

Risk and rollback

There are more small files, and existing open branches need a one-time conversion. A copied row with as above is refused rather than silently referring to a different test. The archive stays in its original order, not a newly inferred chronological order. Duplicate checks do not judge whether measurements are true, and two different descriptions of the same run still need human review.

Reverting the commit restores the old layout. If new fragments have landed by then, first carry their rows into the restored table so evidence is not lost. Do not commit a regenerated combined table alongside the fragments: that would recreate the conflict.

Reviewer notes

Start with docs/tested-configurations.md, the parser's read_row/rows functions and the bare-merge test. The 82 legacy files are text-preserving moves, not new measurements.

Design Trade-off and evidence
Keep one table No workflow change; 13 table conflicts in the pinned sample and 152/153 in the same-base pair experiment
Union attribute Small local convenience, but the bare conflict check still reports all 13; silently keeps contradictory rows and malformed headers
Dated headings Removes this snapshot's next-day insertion conflict, but 137/153 same-base pairs still conflict; relative references also need care when regrouped
Commit generated table with fragments Provides a ready combined page, but contributors still edit one shared output and recreate its insertion conflict
Separate files, uncommitted reading view (chosen) Removes the shared insertion point; costs a one-time branch conversion and a command for the combined view

The maintainer decision is whether to adopt this contributor workflow. No server configuration or release automation is required. A hosted combined view is deliberately outside this change; the individual source files remain readable on the forge.

Issues

See #45

Checklist

For everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description).
  • Rebased on current main.
  • python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
  • Tested in the game, or what wasn't is named under Testing.
  • Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
  • Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. Deterministic conflict counts and source checks above; no paired hardware-timing experiment.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Flags noted, and the switch reference updated (docs/development.md, the README).
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.

New on this forge; not opened on GitHub.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra [email protected]

## Why Independent test results should not force unrelated pull requests to resolve the same table insertion point. This keeps the evidence requirement while giving every measured run its own file. See #45. ## What changed - `docs/tested/`: one three-line Markdown file per run. Each new row is self-contained, with a unique date/topic/tester/run filename. - `docs/tested/legacy/`: all 82 existing rows moved without changing their text or order. The three pre-existing duplicate pairs remain visible, and `as above` keeps its original meaning within the archive. - `scripts/tested_configurations.py`: validates the headers, eight non-empty cells, dates, duplicates and common relative references; assembles a reading view without committing a shared generated table. - `tests/test_tested_configurations.py`: migration digest, malformed/duplicate/interleaved rows, deterministic ordering, escaped pipes, stale-view rejection and real bare-Git merge regressions. Two edits to the same existing row still conflict. - Contributor instructions, the PR template and its example now name the same fragment destination. The description checker itself is unchanged: it reads the description, not the evidence table. ## Flags and settings | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `scripts/tested_configurations.py --output FILE` | New: Writes the assembled Markdown view as UTF-8 | Without an option, prints to stdout | Off by default: choose where to save the view | Reading all results together | Runtime behavior | Windows (should help on: Windows) | | `scripts/tested_configurations.py --check FILE` | New: Fails if the generated view is missing or stale; writes nothing | Not selected | Off by default: only meaningful for an existing view | Checking a saved view | Runtime behavior | Windows (should help on: Windows) | Both are documented in `docs/tested-configurations.md` and the development guide. No game setting changes. ## Measurements <!-- branch-state --> Base: 77f605a25f175fbda9ad131ac6be96bbde85d9c2 Head: 06d7c45637a6343f6fdf3ee59cd556d71ae3b617 Base suite: 509 passed, 0 failed, 19 skipped. Head suite: 510 passed, 0 failed, 19 skipped. <!-- /branch-state --> main: 77f605a25f175fbda9ad131ac6be96bbde85d9c2 (2026-11-13 UTC) this PR: move/fix/tested-rows @ 06d7c45637a6343f6fdf3ee59cd556d71ae3b617 Hardware: Linux 7.2.15-1-omarchy, Git 2.56.0, Python 3.14.8; no GPU work Scene: none; documentation and Git merge behavior Method: pinned heads from the Forgejo API; one deterministic merge per head/design, then every pair of the same row additions; source-test output, not elapsed-time or game-performance measurements How to reproduce: `python -I -B tests/test_tested_configurations.py`; `python -I -B tests/test_check_pr_description.py`; `python -I -B tests/run_all.py --jobs 6 --strict`. For a pinned branch, find its ancestor with `git merge-base BASE HEAD`, then run `git merge-tree --write-tree -z --name-only --no-messages --merge-base=ANCESTOR BASE HEAD` in a bare repository. Repeat after moving only its inserted rows to unique fragment files and restoring that branch's old table to its ancestor's text. | | main @ 77f605a25 | this PR @ 06d7c456 | |---|---|---| | Results-table conflicts, 18 pinned heads | 13 | 0 after row conversion | | Any conflict, same 18 heads | 14 | 2, both unrelated to results | | Concurrent result-only additions, all 153 branch pairs on the same base | 152 conflicting pairs | 0 conflicting pairs | | Existing rows retained in original order | 82 | 82, row-byte digest unchanged | | Whole game-free suite, once per commit | 509 passed, 0 failed, 19 skipped | 510 passed, 0 failed, 19 skipped | ``` Pinned-head replay: table conflicts 13 -> 0; any conflicts 14 -> 2. Same-base pair replay: table 152/153; dated headings 137/153; fragments 0/153. New named test: Ran 14 tests ... OK Description checker tests: Ran 18 tests ... OK Whole suite: 510 passed, 0 failed, 19 skipped. ``` The 16 moved PRs alone are 13 conflicting heads before conversion and 1 afterwards. The two additional PRs are #75 and #78. This is a newer snapshot than the issue comment's historical 14-of-16 result; it does not replace that observation. | PR | Pinned head | Conflicting files before | Conflicting files after row conversion | |---|---|---:|---:| | #47 | `23cc17be6ac3` | 1 | 0 | | #49 | `440fef872d5e` | 1 | 0 | | #50 | `1652ef660014` | 1 | 0 | | #51 | `d5b2ff117905` | 1 | 0 | | #53 | `4ba4554baa59` | 1 | 0 | | #54 | `15813ff4a5ee` | 1 | 0 | | #55 | `a3fdd78b7049` | 0 | 0 | | #56 | `0ce651b604ef` | 1 | 0 | | #57 | `58143b9df475` | 0 | 0 | | #58 | `365b02c32d0f` | 1 | 0 | | #60 | `e8a9913748b3` | 1 | 0 | | #61 | `74f8248ef589` | 0 | 0 | | #66 | `629766eb2063` | 1 | 0 | | #67 | `3969145a3fd5` | 1 | 0 | | #68 | `cca2aebb03ef` | 2 | 1 | | #71 | `e673c7dddb14` | 1 | 0 | | #75 | `a4bd549f215a` | 2 | 2 | | #78 | `f3281c7c7d1b` | 0 | 0 | The remaining files are `docs/development.md` in #68, and the removed `host/hos_update.c` and `tests/test_linux_update_skip.py` in #75. Every non-table branch change was retained. The final replay uses this PR's actual commit, not just a layout sketch. **Conversion is necessary.** Unchanged remote heads do not become mergeable just because this PR merges: all 18 have a one-time layout conflict against the migrated table. Twelve sets of moved rows already pass the new validator. #50, #51, #53, #57, #67 and #68 must spell out their `as above` details. No remote branch was modified in these experiments. **Why not union?** Forgejo 17.0.1's [conflict check](https://codeberg.org/forgejo/forgejo/src/tag/v17.0.1/services/pull/patch.go) runs the command above in the bare base repository. Adding the committed `merge=union` rule left all 13 table conflicts. Reproducing its [sparse merge preparation](https://codeberg.org/forgejo/forgejo/src/tag/v17.0.1/services/pull/merge_prepare.go) also conflicted with merge, squash and rebase when the rule was already shared by both branches. The common non-mergeable status is refused [before the merge style runs](https://codeberg.org/forgejo/forgejo/src/tag/v17.0.1/services/pull/check.go). A full local checkout did honour union. It also returned success while retaining two edits of one row, a duplicate overlapping row and two incompatible headers. Swapping merge direction changed the date order. These are local command reproductions, not a new live server test; the earlier live refusal is already recorded in the issue. The server's Git version and private configuration were not inspected. ## Testing **Tested:** on Linux, the 14-case fragment test, the existing 18-case description-checker test and the whole game-free suite on the exact head above. The migration regression fails against main because the fragments are absent; the merge regression separately reproduces a shared-table conflict and proves distinct files merge in a bare repository. The exact Forgejo merge commands, real-head replay and same-base pair replay are described above. The base suite has no failures; its removed updater test is not carried as a supposed known failure. **Not tested:** Windows execution of the new Python file/path handling; a new live Forgejo merge of fragment PRs; game runs, game builds or frame rates. No game behavior is changed. The static Git experiments are not a measurement of CI time saved. **Needs other testers:** - Windows: run `python -I -B tests/test_tested_configurations.py`, then generate and check a view with a non-ASCII output filename, expecting the tests to pass and the saved view to match (request to open) - Forgejo 17: merge two disposable PRs adding distinct same-date result files, expecting the second to stay mergeable after the first; confirm the new contributor workflow is wanted before doing this (request to open) ## Screenshots Nothing on screen in the game. Each source fragment is an ordinary Markdown table; the combined view is generated on demand. ## Risk and rollback There are more small files, and existing open branches need a one-time conversion. A copied row with `as above` is refused rather than silently referring to a different test. The archive stays in its original order, not a newly inferred chronological order. Duplicate checks do not judge whether measurements are true, and two different descriptions of the same run still need human review. Reverting the commit restores the old layout. If new fragments have landed by then, first carry their rows into the restored table so evidence is not lost. Do not commit a regenerated combined table alongside the fragments: that would recreate the conflict. ## Reviewer notes Start with `docs/tested-configurations.md`, the parser's `read_row`/`rows` functions and the bare-merge test. The 82 legacy files are text-preserving moves, not new measurements. | Design | Trade-off and evidence | |---|---| | Keep one table | No workflow change; 13 table conflicts in the pinned sample and 152/153 in the same-base pair experiment | | Union attribute | Small local convenience, but the bare conflict check still reports all 13; silently keeps contradictory rows and malformed headers | | Dated headings | Removes this snapshot's next-day insertion conflict, but 137/153 same-base pairs still conflict; relative references also need care when regrouped | | Commit generated table with fragments | Provides a ready combined page, but contributors still edit one shared output and recreate its insertion conflict | | Separate files, uncommitted reading view (chosen) | Removes the shared insertion point; costs a one-time branch conversion and a command for the combined view | The maintainer decision is whether to adopt this contributor workflow. No server configuration or release automation is required. A hosted combined view is deliberately outside this change; the individual source files remain readable on the forge. ## Issues See #45 ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [ ] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. Deterministic conflict counts and source checks above; no paired hardware-timing experiment. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Flags noted, and the switch reference updated (docs/development.md, the README). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. --- New on this forge; not opened on GitHub. Assisted-by: GPT-6 Astra Co-Authored-By: GPT-6 Astra <[email protected]>
fix(docs): give tested runs independent result files
Some checks are pending
Linux AppImage / appimage (pull_request) Blocked by required conditions
Details
Windows builder / Longshot Builder zip (pull_request) Blocked by required conditions
Details
Windows builder / Windows Vulkan back end (llvm-mingw) (pull_request) Blocked by required conditions
Details
Windows builder / dav1d for Windows (llvm-mingw) (pull_request) Blocked by required conditions
Details
CI / Conventional Commit title (pull_request) Blocked by required conditions
Details
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
Details
CI / Python source checks (windows-2022, part 1/2) (pull_request) Blocked by required conditions
Details
CI / Python source checks (windows-2022, part 2/2) (pull_request) Blocked by required conditions
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
Details
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Details
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Details
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Details
PR description / Description follows the template (pull_request_target) Successful in 2s
Details
06d7c45637 
Keep one validated Markdown fragment per run and assemble the reading view without committing it. Preserve every existing row and its order; retain known historical duplicates explicitly. See #45.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>
Some checks are pending
Linux AppImage / appimage (pull_request) Blocked by required conditions
Details
Windows builder / Longshot Builder zip (pull_request) Blocked by required conditions
Details
Windows builder / Windows Vulkan back end (llvm-mingw) (pull_request) Blocked by required conditions
Details
Windows builder / dav1d for Windows (llvm-mingw) (pull_request) Blocked by required conditions
Details
CI / Conventional Commit title (pull_request) Blocked by required conditions
Details
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
Details
CI / Python source checks (windows-2022, part 1/2) (pull_request) Blocked by required conditions
Details
CI / Python source checks (windows-2022, part 2/2) (pull_request) Blocked by required conditions
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
Details
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Details
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Details
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Details
PR description / Description follows the template (pull_request_target) Successful in 2s
Details
This pull request has changes conflicting with the target branch.
  • docs/tested-configurations.md
  • tests/pr_description/good.md
Some workflows are waiting to be reviewed.
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/tested-rows:mooki-fix/tested-rows'
git switch 'mooki-fix/tested-rows'
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
accessibility
Barrier affecting people with disabilities

  
autorelease: pending

   
autorelease: tagged

   
bug
Something isn't working

  
dependencies
Pull requests that update a dependency file

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
github_actions
Pull requests that update GitHub Actions code

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
needs measurements
Waiting for before/after measurements

  
question
Further information is requested

  
testing wanted
Wanted: a test on other hardware (posts a test request in #test-requests)

  
wontfix
This will not be worked on

No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
needs measurements
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!96
No description provided.