docs(forge): every pull request description starts with a plain TL;DR and a Breakdown; the check suggests them where they are missing #1470

Merged
mooki merged 2 commits from mooki/Longshot:docs/pr-template-tldr into main 2026-11-17 21:17:47 +00:00
Member

Type: docs · Risk: low · Tested on: Linux (Arch), no GPU: it changes no game file · Needs: the forge's new pull request page, after merge

TL;DR

A pull request's description is long, and someone who is not reading the code has to dig through it to learn what a change is for. The template now opens with a TL;DR, a few plain sentences, and a Breakdown, short bullets, and the description check asks for them to be filled in when they are there. When they are missing it only suggests them, so the open pull requests keep passing: read over all of them, the new check adds no error. The check's new tests failed before the change and pass after it, and the whole suite gives main's result.

Breakdown

  • Who it helps: anyone who reads a pull request to learn what it is for: players, testers and maintainers who are not reading the diff.
  • What was wrong: the template starts with Why, a long section, so the short story of a pull request (who it helps, what changes, how it was checked) was scattered or missing, and the check could not ask for it.
  • What changes: the template gets a TL;DR and a Breakdown right after the status line, each with a comment that explains it and a short example. The check reads them: kept, they must be filled in (no placeholder, a TL;DR of 120 words or fewer, a Breakdown with bullets and none empty); left out, each is one warning that suggests it. CONTRIBUTING.md lists them.
  • What does not change: the status line is still the first line; no other section or rule changes; a description without the two sections still passes, so no open pull request fails for lacking them; the check's functions keep their shapes, so Tooling's forge.py needs no change.
  • How it was checked: 15 of the check's 51 tests fail on main's check and template and all 51 pass here; six one-line breakages of the new rules are each caught; the whole suite gives main's result (674 passed, 38 failed, 17 skipped, the same 38 names); the new check read all 334 open descriptions and adds no error (an earlier read found two TL;DRs over the limit, #1466 and #68, since cut to 115 words).
  • What is still open: how the forge's new pull request page shows the two sections (after merge); Tooling's docs/forge.md, in another repository, lists what the check refuses and does not name these two; Windows (plain text rules in Python, not run).
  • Risk: low. A text check with a warning for what is missing; what it newly refuses is an over-long TL;DR, an empty Breakdown bullet or a placeholder left in a section that was kept.

Why

A description opens, after the status line, with Why, and under it come the reproduction steps, the measurements, the testing and the checklist. A reader who only wants to know what a pull request does, who it helps and whether it was checked has to read it all and sum it up in their head. I want every pull request to start with a human-readable TL;DR and a breakdown, and the template to carry them; #1462's description shows the form.

This builds on #1454 (merged today), which made "ready for review" one list and put a complete description in it (CONTRIBUTING.md, "Ready for review: out of WIP", item 5): this adds the part of that description a reader reaches first. I searched the tracker for "TL;DR", "TLDR", "pull request template", "PR description" and "breakdown": no issue asks for it, so there is nothing to link; the pull requests that touch the same files are under Overlaps.

What changed

The template gets a TL;DR and a Breakdown right after the status line, and the description check reads them. Kept, they must be filled in; left out, they only draw a warning, so no open pull request starts to fail for lacking them.

  • template: .forgejo/pull_request_template.md has ## TL;DR (two to four plain sentences, at most 120 words: what was wrong or missing, what this changes, the headline result and how it was checked) and ## Breakdown (seven bullets: who it helps, what was wrong, what changes, what does not change, how it was checked, what is still open, the risk), each with an HTML comment that explains it and a short example, in the template's existing style. Why's comment no longer says "first".
  • check: scripts/check_pr_description.py has SUMMARY, TLDR_WORDS = 120, summary() (finds the section by its heading without case or punctuation, so TLDR counts), tldr_problems() and breakdown_problems(); review() runs them after the status line's checks and before Why. Placeholders are the template's own <...> stand-ins, found the way every other section's are. A missing section adds one warning to the returned warnings; the returned (problems, warnings) shape, problems(), TEMPLATE, keep_applicable() and without_reproduce() (what Tooling's forge.py calls) are as they were. status() is untouched, so a TL;DR above the status line is "no status line".
  • docs: CONTRIBUTING.md, "Commits and pull requests", step 3's list of sections names the two. docs/workflow.md and docs/agent-pull-requests.md only link the template and CONTRIBUTING.md (they list no sections), so they are unchanged.
  • tests: tests/test_check_pr_description.py has a new Summary class (10 tests) and updated samples; tests/pr_description/good.md, the sample of a good description, has both sections. The "older description" sample is good.md less the two sections, taken in the test, so it follows the template when that changes.

Unchanged: every other section's rules, the checklist, the WIP notice, the workflow (.forgejo/workflows/pr-description.yml runs main's copy of the script on every pull request, so once this merges the new rules apply to every open description at its next edit).

Measurements

Verdict: the new tests fail on main's check and template (15 of 51) and pass here (51 of 51); the whole suite is main's (674 passed, 38 failed, 17 skipped, the same 38 failing names); read over all 334 open descriptions the new check adds no error and drops none, and its warnings reach every description that has neither section; its time over all those descriptions is the same within the spread.

Hardware: Linux (Arch), kernel 7.2.15, i9-13900K (32 threads), 93 GB; no GPU, no game
Scene: none: a Python check on text; the open pull requests' real descriptions read from the forge
Method: tests/test_check_pr_description.py at the tests-only commit and at the head; tests/run_all.py --jobs 6 --strict on this head and on main; a scan script runs main's check with main's template and this branch's check with its template over every open description (the call --pr makes, without the diff, which the new rules do not read); the check's cost is five interleaved timed passes of each check over those descriptions
How to reproduce: python -I -B tests/test_check_pr_description.py; python -I -B tests/run_all.py --jobs 6 --strict (RX_CPM set to the dependency cache that holds the Vulkan and SPIR-V headers); python -I scan_open_prs.py --old <main checkout> --new <this checkout>

Before: main @ 257c54911 After: this PR @ a51766cb9
tests/test_check_pr_description.py main's own file: 41 of 41 pass; with this PR's tests added (cf3590fa9): 36 of 51 pass, 15 fail (9 failures, 6 errors) 51 of 51 pass
whole suite (712 checks) 674 passed, 38 failed, 17 skipped 674 passed, 38 failed, 17 skipped (the same 38 names)
open descriptions read 334 334
open descriptions failing the check 207 207
errors added by the new rules - 0 added, 0 dropped (an earlier read, at 20:41 UTC, found 2: #1466's TL;DR of 191 words and #68's of 155; both were cut to 115 words)
descriptions that get the TL;DR / Breakdown warning 0 / 0 331 / 331 (every description that has neither section)
the check's time over 335 descriptions (5 interleaved passes, 1-minute load average 11) 0.27 to 0.29 s 0.28 to 0.29 s
Ran 51 tests in 0.282s  FAILED (failures=9, errors=6)       tests-only commit cf3590fa9 on main's check and template
Ran 51 tests in 0.302s  OK                                    head a51766cb9
674 passed, 38 failed, 17 skipped in 203 s (linux, 6 at once)   this head, the second full run
674 passed, 38 failed, 17 skipped in 200 s (linux, 6 at once)   main
open pull requests read: 334
failing the old check: 207; failing the new check: 207
errors the new check adds: 0
errors the new check drops: 0
new warnings (descriptions that get each):
  331  no ## Breakdown
  331  no ## TL;DR

Flags and settings

None.

How it was tested

Everything here was run by an AI agent (Claude Sonnet 5.5) working for me; the commands are given so anyone can rerun them. No game was run: nothing here is read by the game (.forgejo/, a script under scripts/, tests, a doc).

1. What was looked for

  1. Where the check runs and what it reads. .forgejo/workflows/pr-description.yml runs main's scripts/check_pr_description.py --pr N on every pull request event, with main's template, so after a merge the new rules apply to every open description at its next edit. forge.py (Tooling) loads the same file and calls review, problems, keep_applicable and without_reproduce and reads TEMPLATE: all four and the (problems, warnings) shape are unchanged. A rule that errors on a missing section would turn every open description red, so a missing section is a warning.
  2. What the open descriptions look like. The forge's public pull request list, read page by page (GET /api/v1/repos/MagicBeanDev/Longshot/pulls?state=open&limit=50&page=N until an empty page), gave 337 open pull requests at the first read, 338 at the second, 335 at the next two and 334 at the last, at 21:02 UTC (they merge and close while this is written). At the first read one description had a TL;DR or Breakdown heading (#1462); by the last, three (#1462, #68, #1466), as TL;DRs began to be added to open pull requests today. The earlier reads (the last at 20:41 UTC) found #1466 at 191 words and #68 at 155: the only two errors the new rules added. Both were then cut to 115 words, and the read at 21:02 UTC adds none.

2. Game-free checks (anyone can rerun them)

  • Environment: Arch Linux, kernel 7.2.15-1-omarchy, i9-13900K (32 threads), 93 GB, no GPU used. Python 3.12.16 (the pinned build Python, python -I -B), git 2.56.0, clang++ 23.1.3 and g++ 16.2.1 (the suite's compiled tests). Base: main 257c54911 (forge main, still current when fetched at 20:54 UTC on 17 November, after #1454); head a51766cb9, two commits: cf3590fa9 (the tests alone) and a51766cb9 (the template, the check, CONTRIBUTING.md). 17 November 2026, 19:40 to 21:05 UTC. Main moved three times while this was written (#718; #1459 with two others; none touches a file of the check, the template, CONTRIBUTING.md or the pull request tests): the two commits were rebased each time, git range-diff shows both unchanged, and the tests, the suite and the scan were rerun on the final base; the cost passes below were timed on the head before the last rebase (the same patch). The machine runs other builds and checks at the same time: the 1-minute load average was 11 to 23 when read during the runs (the 15-minute one up to 37); nothing here is a timing claim except the one table row that says so.
  • How the checks were run: through the workbench's idle-class gate (https://github.com/mooki000/ls-workbench: it takes one of three test slots and runs the job in the idle scheduling class with core dumps off), from a clean git worktree of each commit, never from the working tree being edited.
  • Tests first, red then green. cf3590fa9 holds only the tests and good.md. python -I -B tests/test_check_pr_description.py in a clean checkout of it (main's check and template): Ran 51 tests ... FAILED (failures=9, errors=6). The 15 are: the sample and template tests that need a TL;DR section to exist (test_the_fix_sample_passes, test_a_docs_change_with_one_line_passes, test_contributings_example_passes, test_the_bare_template_fails_on_every_section, test_the_sections_follow_the_order, test_without_reproduce) and the new Summary tests (placeholders, length, bullets, empty bullets, the heading's spelling, warnings and the command's output, the template's order). Three of them error on a name that does not exist yet (check.SUMMARY, check.TLDR_WORDS), one on KeyError: 'Breakdown' and two on the template having no TL;DR section for the sample helpers to fill; nine fail on behaviour (a missing section gives no warning, an empty TL;DR is accepted, the template's sections are not in the new order). The tenth Summary test, test_the_status_line_still_comes_first, passes on main on purpose: the status line rule is not changed. At the head: Ran 51 tests ... OK. Main's own copy of the file (41 tests) passes on main.
  • Mutation checks. In a throwaway copy of the head, six one-line changes to scripts/check_pr_description.py, each followed by the same test file through the gate; each must fail: the limit raised to 121 (test_the_tldr_is_short, test_the_template_has_them_right_after_the_status_line); a missing section made an error (test_left_out_is_a_warning_never_an_error, test_the_command_passes_and_says_so); the placeholder check skipped for the two sections (three tests); the empty-bullet check off (test_a_bullet_is_not_left_empty); the no-bullets check off (test_the_breakdown_has_bullets); the heading read exactly, so TLDR is missed (test_the_heading_may_be_spelled_tldr). All six exit 1 on an assertion, none on a syntax error.
  • Whole suite. python -I -B tests/run_all.py --jobs 6 --strict through the gate, with RX_CPM set to the dependency cache (a first run without it failed 94 tests on both trees: --strict fails a test whose Vulkan or SPIR-V headers it cannot find; with it, the 38 below), on this head and then on main 257c54911, one after the other: 674 passed / 38 failed / 17 skipped (712 checks) on both, and the same 38 names. test_crash_record.py timed out on a 10 s wait for a killed child in the first full run of the head, twice over the work (once on the previous base, once on this one), under heavy load (the 15-minute load average was 30 to 33); alone it passes on the head and on main each time it was tried, and the next full run of the head has it passing. The 38 are main's own on this machine, none in or near the description check (tests/test_check_pr_description.py passes on both): test_bug_report_defaults.py, test_cpu_report.py, test_crt_options.py, test_damage_debug_menu.py, test_display_startup_failure.py, test_dlss_feature_size.py, test_failed_compile_keeps_objects.py, test_framegen_preset.py, test_fsr_uniform_ownership.py, test_guest_imported.py, test_hud_layout_row.py, test_import_reach.py, test_invalidate_vcore.py, test_linux_preflight.py, test_linux_pressure.py, test_linux_update.py, test_log_open_failure.py, test_mirror_budget.py, test_missing_nup.py, test_mods_size_table.py, test_nonuniform_image_arrays.py, test_parallel_init_order.py, test_pending_guest_import.py, test_pipeline_play_cache.py, test_present_thread.py, test_profiling_mode.py, test_readback_timeout_source.py, test_replay_write_order.py, test_rumble_menu.py, test_run_all_changed.py, test_slow_frames.py, test_staging_burst.py, test_target_image_pool.py, test_upscaler_round_trip.py, test_upscaler_rows.py, test_vcore_escape.py, test_vk_occlusion_queries.py, test_walk_key.py.
  • The live descriptions. scan_open_prs.py (attached to the first comment as scan_open_prs.py.txt, about 80 lines, standard library only, public reads only) loads two copies of the check (each with its template) from two checkouts, reads every open description with the forge's public API and runs review(body, merge_base=..., title=...) of each on it: the call --pr makes, without the diff, which the new rules do not read. It prints how many fail with each, every error the new check adds or drops, and how many descriptions get each new warning (its output is the block above). The three descriptions that have the sections (#1462, #1466, #68) were also run through the new check one by one at 21:03 UTC: their TL;DRs are 93, 115 and 115 words and the TL;DR and Breakdown raise nothing; the one error #1462 and #68 still have is an unticked box of #1454's Everyone list, which main's check reports the same way.
  • The cost. The same 335 descriptions through each check, five passes each, interleaved (old, new, old, new...): 0.27, 0.27, 0.27, 0.27, 0.29 s and 0.28, 0.28, 0.29, 0.29, 0.29 s, about 0.8 ms per description; the 1-minute load average was 11 at the last pass.
  • Overlaps. Every open pull request's pulls/N/files (334 of 335 at that read: pulls/768/files answered null, an incomplete answer, so its head was merged on its own), then git merge-tree --write-tree --name-only <base> <head> of each head that changes one of this branch's files, against main and against this branch.
  • Not run: the Windows-target fixture check: no C or C++ file changed, and the Python change reads no path, file or process.

3. In the game

Not run: no file the game runs changed, and pr create asks for a game run only for gpu/, host/, nx2/, assets/ and third_party/.

4. What was not tested

See "Not tested" under Testing.

Testing

Tested: on this PC (Linux, no GPU), the check and its 51 tests at the tests-only commit (15 fail on main's check and template) and at the head (all pass); six deliberate breakages of the new rules, each caught; the whole suite on the head and on main (the same 674 passed, 38 failed, 17 skipped, the same names); the new check against all 334 open descriptions (no error added; 331 of 334 get the two suggestions) and against the live descriptions of #1462, #1466 and #68, the three that have the sections (their TL;DRs of 93, 115 and 115 words raise nothing); a trial merge of this branch with each open pull request that changes the same files (no conflict added; with #1454, which merged first, the 51 tests passed on the merge of this branch with its head before the rebase).

Not tested: how the forge's new pull request page shows the template (the headings, the stand-ins, the comments hidden): the page cannot be opened for a branch that is not on main yet; forge.py pr create against the new template (it opens a pull request: not run; its calls into the check were read, and their shapes did not change); the workflow itself on the forge (it runs main's copy, so it runs on the first pull request event after the merge); Windows (the check is plain Python text rules: not run); a feat or perf description made from the new template (the tests use a fix, a perf and a docs sample).

Must be tested:

  • the forge's new pull request page, any browser, after merge: start a new pull request from any branch of a fork and look at the description box, expecting the status line, then the TL;DR and Breakdown headings with their stand-ins and no comment text showing (by hand: look at the box, and close it without creating anything)

Risks

What could break: a description that keeps a TL;DR over 120 words, a Breakdown without bullets or with an empty bullet, or a placeholder in either, is refused when it is marked ready, and the workflow flags it on its next edit. No open description is in that case now: two were when the check first read them (#1466's TL;DR was 191 words and #68's 155, both added today) and both were cut to 115 words; a looser limit is one constant, TLDR_WORDS. A ## TL;DR or ## Breakdown heading used for something else is read as these; none of the 337 descriptions read at the first look had one. Cost: the check reads two short sections more: 0.28 to 0.29 s against 0.27 to 0.29 s over 335 descriptions (about 0.8 ms each), on a loaded PC, so no measurable difference.

Turning it off: revert the commit, or delete the two sections from a description (it then only warns).

Compatibility: nothing changes for a player; a contributor who starts a pull request from the new template sees two more sections to fill in, and a description from the old template passes with two warnings.

Overlaps

Read from pulls/N/files of the open pull requests and trial merges (git merge-tree --write-tree) of each head against main and against this branch, conflict hunks read where there were any (done on main dcb7b47c2; the commits added to main since touch none of this branch's files). #1454, which changed the same template, check, tests, good.md and CONTRIBUTING.md, merged while this was written: this branch was rebased onto it (the range-diff of the two commits shows only changed context lines) and the tests and the suite were rerun on the result. Fourteen open pull requests change CONTRIBUTING.md and nothing else of this branch's files (#29, #56, #67, #68, #144, #204, #235, #236, #284, #405, #685, #860, #893, #1145): eight merge cleanly with main and with this branch; six conflict with main already (#29 in CONTRIBUTING.md and tests/run_all.py, #204, #236, #284, #405, #685 in other files) and this branch adds no conflict to any of them. #768's file list was not readable (null); its head merges cleanly with both.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named): 38 tests fail on both main and this PR, the same names, none in the description check: test_bug_report_defaults.py, test_cpu_report.py, test_crt_options.py, test_damage_debug_menu.py, test_display_startup_failure.py, test_dlss_feature_size.py, test_failed_compile_keeps_objects.py, test_framegen_preset.py, test_fsr_uniform_ownership.py, test_guest_imported.py, test_hud_layout_row.py, test_import_reach.py, test_invalidate_vcore.py, test_linux_preflight.py, test_linux_pressure.py, test_linux_update.py, test_log_open_failure.py, test_mirror_budget.py, test_missing_nup.py, test_mods_size_table.py, test_nonuniform_image_arrays.py, test_parallel_init_order.py, test_pending_guest_import.py, test_pipeline_play_cache.py, test_present_thread.py, test_profiling_mode.py, test_readback_timeout_source.py, test_replay_write_order.py, test_rumble_menu.py, test_run_all_changed.py, test_slow_frames.py, test_staging_burst.py, test_target_image_pool.py, test_upscaler_round_trip.py, test_upscaler_rows.py, test_vcore_escape.py, test_vk_occlusion_queries.py, test_walk_key.py.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A check under tests/ that reads changed source updated in this PR, not weakened (tests/test_check_pr_description.py reads the template, the check and CONTRIBUTING.md; its samples and test_the_sections_follow_the_order, test_without_reproduce and test_refusals follow the new sections, and nothing was removed).
  • Docs updated.

New on this forge; not opened on GitHub.

**Type:** docs · **Risk:** low · **Tested on:** Linux (Arch), no GPU: it changes no game file · **Needs:** the forge's new pull request page, after merge ## TL;DR A pull request's description is long, and someone who is not reading the code has to dig through it to learn what a change is for. The template now opens with a TL;DR, a few plain sentences, and a Breakdown, short bullets, and the description check asks for them to be filled in when they are there. When they are missing it only suggests them, so the open pull requests keep passing: read over all of them, the new check adds no error. The check's new tests failed before the change and pass after it, and the whole suite gives main's result. ## Breakdown - **Who it helps:** anyone who reads a pull request to learn what it is for: players, testers and maintainers who are not reading the diff. - **What was wrong:** the template starts with Why, a long section, so the short story of a pull request (who it helps, what changes, how it was checked) was scattered or missing, and the check could not ask for it. - **What changes:** the template gets a TL;DR and a Breakdown right after the status line, each with a comment that explains it and a short example. The check reads them: kept, they must be filled in (no placeholder, a TL;DR of 120 words or fewer, a Breakdown with bullets and none empty); left out, each is one warning that suggests it. CONTRIBUTING.md lists them. - **What does not change:** the status line is still the first line; no other section or rule changes; a description without the two sections still passes, so no open pull request fails for lacking them; the check's functions keep their shapes, so Tooling's `forge.py` needs no change. - **How it was checked:** 15 of the check's 51 tests fail on main's check and template and all 51 pass here; six one-line breakages of the new rules are each caught; the whole suite gives main's result (674 passed, 38 failed, 17 skipped, the same 38 names); the new check read all 334 open descriptions and adds no error (an earlier read found two TL;DRs over the limit, #1466 and #68, since cut to 115 words). - **What is still open:** how the forge's new pull request page shows the two sections (after merge); Tooling's docs/forge.md, in another repository, lists what the check refuses and does not name these two; Windows (plain text rules in Python, not run). - **Risk:** low. A text check with a warning for what is missing; what it newly refuses is an over-long TL;DR, an empty Breakdown bullet or a placeholder left in a section that was kept. ## Why A description opens, after the status line, with Why, and under it come the reproduction steps, the measurements, the testing and the checklist. A reader who only wants to know what a pull request does, who it helps and whether it was checked has to read it all and sum it up in their head. I want every pull request to start with a human-readable TL;DR and a breakdown, and the template to carry them; #1462's description shows the form. This builds on #1454 (merged today), which made "ready for review" one list and put a complete description in it (CONTRIBUTING.md, "Ready for review: out of WIP", item 5): this adds the part of that description a reader reaches first. I searched the tracker for "TL;DR", "TLDR", "pull request template", "PR description" and "breakdown": no issue asks for it, so there is nothing to link; the pull requests that touch the same files are under Overlaps. ## What changed The template gets a TL;DR and a Breakdown right after the status line, and the description check reads them. Kept, they must be filled in; left out, they only draw a warning, so no open pull request starts to fail for lacking them. - template: `.forgejo/pull_request_template.md` has `## TL;DR` (two to four plain sentences, at most 120 words: what was wrong or missing, what this changes, the headline result and how it was checked) and `## Breakdown` (seven bullets: who it helps, what was wrong, what changes, what does not change, how it was checked, what is still open, the risk), each with an HTML comment that explains it and a short example, in the template's existing style. Why's comment no longer says "first". - check: `scripts/check_pr_description.py` has `SUMMARY`, `TLDR_WORDS = 120`, `summary()` (finds the section by its heading without case or punctuation, so `TLDR` counts), `tldr_problems()` and `breakdown_problems()`; `review()` runs them after the status line's checks and before Why. Placeholders are the template's own `<...>` stand-ins, found the way every other section's are. A missing section adds one warning to the returned warnings; the returned `(problems, warnings)` shape, `problems()`, `TEMPLATE`, `keep_applicable()` and `without_reproduce()` (what Tooling's `forge.py` calls) are as they were. `status()` is untouched, so a TL;DR above the status line is "no status line". - docs: CONTRIBUTING.md, "Commits and pull requests", step 3's list of sections names the two. `docs/workflow.md` and `docs/agent-pull-requests.md` only link the template and CONTRIBUTING.md (they list no sections), so they are unchanged. - tests: `tests/test_check_pr_description.py` has a new `Summary` class (10 tests) and updated samples; `tests/pr_description/good.md`, the sample of a good description, has both sections. The "older description" sample is good.md less the two sections, taken in the test, so it follows the template when that changes. Unchanged: every other section's rules, the checklist, the WIP notice, the workflow (`.forgejo/workflows/pr-description.yml` runs main's copy of the script on every pull request, so once this merges the new rules apply to every open description at its next edit). ## Measurements **Verdict:** the new tests fail on main's check and template (15 of 51) and pass here (51 of 51); the whole suite is main's (674 passed, 38 failed, 17 skipped, the same 38 failing names); read over all 334 open descriptions the new check adds no error and drops none, and its warnings reach every description that has neither section; its time over all those descriptions is the same within the spread. Hardware: Linux (Arch), kernel 7.2.15, i9-13900K (32 threads), 93 GB; no GPU, no game Scene: none: a Python check on text; the open pull requests' real descriptions read from the forge Method: `tests/test_check_pr_description.py` at the tests-only commit and at the head; `tests/run_all.py --jobs 6 --strict` on this head and on main; a scan script runs main's check with main's template and this branch's check with its template over every open description (the call `--pr` makes, without the diff, which the new rules do not read); the check's cost is five interleaved timed passes of each check over those descriptions How to reproduce: `python -I -B tests/test_check_pr_description.py`; `python -I -B tests/run_all.py --jobs 6 --strict` (`RX_CPM` set to the dependency cache that holds the Vulkan and SPIR-V headers); `python -I scan_open_prs.py --old <main checkout> --new <this checkout>` | | Before: main @ 257c54911 | After: this PR @ a51766cb9 | |---|---|---| | tests/test_check_pr_description.py | main's own file: 41 of 41 pass; with this PR's tests added (cf3590fa9): 36 of 51 pass, 15 fail (9 failures, 6 errors) | 51 of 51 pass | | whole suite (712 checks) | 674 passed, 38 failed, 17 skipped | 674 passed, 38 failed, 17 skipped (the same 38 names) | | open descriptions read | 334 | 334 | | open descriptions failing the check | 207 | 207 | | errors added by the new rules | - | 0 added, 0 dropped (an earlier read, at 20:41 UTC, found 2: #1466's TL;DR of 191 words and #68's of 155; both were cut to 115 words) | | descriptions that get the TL;DR / Breakdown warning | 0 / 0 | 331 / 331 (every description that has neither section) | | the check's time over 335 descriptions (5 interleaved passes, 1-minute load average 11) | 0.27 to 0.29 s | 0.28 to 0.29 s | ``` Ran 51 tests in 0.282s FAILED (failures=9, errors=6) tests-only commit cf3590fa9 on main's check and template Ran 51 tests in 0.302s OK head a51766cb9 674 passed, 38 failed, 17 skipped in 203 s (linux, 6 at once) this head, the second full run 674 passed, 38 failed, 17 skipped in 200 s (linux, 6 at once) main open pull requests read: 334 failing the old check: 207; failing the new check: 207 errors the new check adds: 0 errors the new check drops: 0 new warnings (descriptions that get each): 331 no ## Breakdown 331 no ## TL;DR ``` ### Flags and settings None. ## How it was tested Everything here was run by an AI agent (Claude Sonnet 5.5) working for me; the commands are given so anyone can rerun them. No game was run: nothing here is read by the game (`.forgejo/`, a script under `scripts/`, tests, a doc). ### 1. What was looked for 1. **Where the check runs and what it reads.** `.forgejo/workflows/pr-description.yml` runs main's `scripts/check_pr_description.py --pr N` on every pull request event, with main's template, so after a merge the new rules apply to every open description at its next edit. `forge.py` (Tooling) loads the same file and calls `review`, `problems`, `keep_applicable` and `without_reproduce` and reads `TEMPLATE`: all four and the `(problems, warnings)` shape are unchanged. A rule that errors on a missing section would turn every open description red, so a missing section is a warning. 2. **What the open descriptions look like.** The forge's public pull request list, read page by page (`GET /api/v1/repos/MagicBeanDev/Longshot/pulls?state=open&limit=50&page=N` until an empty page), gave 337 open pull requests at the first read, 338 at the second, 335 at the next two and 334 at the last, at 21:02 UTC (they merge and close while this is written). At the first read one description had a `TL;DR` or `Breakdown` heading (#1462); by the last, three (#1462, #68, #1466), as TL;DRs began to be added to open pull requests today. The earlier reads (the last at 20:41 UTC) found #1466 at 191 words and #68 at 155: the only two errors the new rules added. Both were then cut to 115 words, and the read at 21:02 UTC adds none. ### 2. Game-free checks (anyone can rerun them) - **Environment:** Arch Linux, kernel 7.2.15-1-omarchy, i9-13900K (32 threads), 93 GB, no GPU used. Python 3.12.16 (the pinned build Python, `python -I -B`), git 2.56.0, clang++ 23.1.3 and g++ 16.2.1 (the suite's compiled tests). Base: main `257c54911` (forge main, still current when fetched at 20:54 UTC on 17 November, after #1454); head `a51766cb9`, two commits: `cf3590fa9` (the tests alone) and `a51766cb9` (the template, the check, CONTRIBUTING.md). 17 November 2026, 19:40 to 21:05 UTC. Main moved three times while this was written (#718; #1459 with two others; none touches a file of the check, the template, CONTRIBUTING.md or the pull request tests): the two commits were rebased each time, `git range-diff` shows both unchanged, and the tests, the suite and the scan were rerun on the final base; the cost passes below were timed on the head before the last rebase (the same patch). The machine runs other builds and checks at the same time: the 1-minute load average was 11 to 23 when read during the runs (the 15-minute one up to 37); nothing here is a timing claim except the one table row that says so. - **How the checks were run:** through the workbench's idle-class gate (https://github.com/mooki000/ls-workbench: it takes one of three test slots and runs the job in the idle scheduling class with core dumps off), from a clean `git worktree` of each commit, never from the working tree being edited. - **Tests first, red then green.** `cf3590fa9` holds only the tests and `good.md`. `python -I -B tests/test_check_pr_description.py` in a clean checkout of it (main's check and template): `Ran 51 tests ... FAILED (failures=9, errors=6)`. The 15 are: the sample and template tests that need a TL;DR section to exist (`test_the_fix_sample_passes`, `test_a_docs_change_with_one_line_passes`, `test_contributings_example_passes`, `test_the_bare_template_fails_on_every_section`, `test_the_sections_follow_the_order`, `test_without_reproduce`) and the new `Summary` tests (placeholders, length, bullets, empty bullets, the heading's spelling, warnings and the command's output, the template's order). Three of them error on a name that does not exist yet (`check.SUMMARY`, `check.TLDR_WORDS`), one on `KeyError: 'Breakdown'` and two on the template having no TL;DR section for the sample helpers to fill; nine fail on behaviour (a missing section gives no warning, an empty TL;DR is accepted, the template's sections are not in the new order). The tenth `Summary` test, `test_the_status_line_still_comes_first`, passes on main on purpose: the status line rule is not changed. At the head: `Ran 51 tests ... OK`. Main's own copy of the file (41 tests) passes on main. - **Mutation checks.** In a throwaway copy of the head, six one-line changes to `scripts/check_pr_description.py`, each followed by the same test file through the gate; each must fail: the limit raised to 121 (`test_the_tldr_is_short`, `test_the_template_has_them_right_after_the_status_line`); a missing section made an error (`test_left_out_is_a_warning_never_an_error`, `test_the_command_passes_and_says_so`); the placeholder check skipped for the two sections (three tests); the empty-bullet check off (`test_a_bullet_is_not_left_empty`); the no-bullets check off (`test_the_breakdown_has_bullets`); the heading read exactly, so `TLDR` is missed (`test_the_heading_may_be_spelled_tldr`). All six exit 1 on an assertion, none on a syntax error. - **Whole suite.** `python -I -B tests/run_all.py --jobs 6 --strict` through the gate, with `RX_CPM` set to the dependency cache (a first run without it failed 94 tests on both trees: `--strict` fails a test whose Vulkan or SPIR-V headers it cannot find; with it, the 38 below), on this head and then on main `257c54911`, one after the other: 674 passed / 38 failed / 17 skipped (712 checks) on both, and the same 38 names. `test_crash_record.py` timed out on a 10 s wait for a killed child in the first full run of the head, twice over the work (once on the previous base, once on this one), under heavy load (the 15-minute load average was 30 to 33); alone it passes on the head and on main each time it was tried, and the next full run of the head has it passing. The 38 are main's own on this machine, none in or near the description check (`tests/test_check_pr_description.py` passes on both): test_bug_report_defaults.py, test_cpu_report.py, test_crt_options.py, test_damage_debug_menu.py, test_display_startup_failure.py, test_dlss_feature_size.py, test_failed_compile_keeps_objects.py, test_framegen_preset.py, test_fsr_uniform_ownership.py, test_guest_imported.py, test_hud_layout_row.py, test_import_reach.py, test_invalidate_vcore.py, test_linux_preflight.py, test_linux_pressure.py, test_linux_update.py, test_log_open_failure.py, test_mirror_budget.py, test_missing_nup.py, test_mods_size_table.py, test_nonuniform_image_arrays.py, test_parallel_init_order.py, test_pending_guest_import.py, test_pipeline_play_cache.py, test_present_thread.py, test_profiling_mode.py, test_readback_timeout_source.py, test_replay_write_order.py, test_rumble_menu.py, test_run_all_changed.py, test_slow_frames.py, test_staging_burst.py, test_target_image_pool.py, test_upscaler_round_trip.py, test_upscaler_rows.py, test_vcore_escape.py, test_vk_occlusion_queries.py, test_walk_key.py. - **The live descriptions.** `scan_open_prs.py` (attached to the first comment as `scan_open_prs.py.txt`, about 80 lines, standard library only, public reads only) loads two copies of the check (each with its template) from two checkouts, reads every open description with the forge's public API and runs `review(body, merge_base=..., title=...)` of each on it: the call `--pr` makes, without the diff, which the new rules do not read. It prints how many fail with each, every error the new check adds or drops, and how many descriptions get each new warning (its output is the block above). The three descriptions that have the sections (#1462, #1466, #68) were also run through the new check one by one at 21:03 UTC: their TL;DRs are 93, 115 and 115 words and the TL;DR and Breakdown raise nothing; the one error #1462 and #68 still have is an unticked box of #1454's Everyone list, which main's check reports the same way. - **The cost.** The same 335 descriptions through each check, five passes each, interleaved (old, new, old, new...): 0.27, 0.27, 0.27, 0.27, 0.29 s and 0.28, 0.28, 0.29, 0.29, 0.29 s, about 0.8 ms per description; the 1-minute load average was 11 at the last pass. - **Overlaps.** Every open pull request's `pulls/N/files` (334 of 335 at that read: `pulls/768/files` answered `null`, an incomplete answer, so its head was merged on its own), then `git merge-tree --write-tree --name-only <base> <head>` of each head that changes one of this branch's files, against main and against this branch. - **Not run:** the Windows-target fixture check: no C or C++ file changed, and the Python change reads no path, file or process. ### 3. In the game Not run: no file the game runs changed, and `pr create` asks for a game run only for `gpu/`, `host/`, `nx2/`, `assets/` and `third_party/`. ### 4. What was not tested See "Not tested" under Testing. ## Testing **Tested:** on this PC (Linux, no GPU), the check and its 51 tests at the tests-only commit (15 fail on main's check and template) and at the head (all pass); six deliberate breakages of the new rules, each caught; the whole suite on the head and on main (the same 674 passed, 38 failed, 17 skipped, the same names); the new check against all 334 open descriptions (no error added; 331 of 334 get the two suggestions) and against the live descriptions of #1462, #1466 and #68, the three that have the sections (their TL;DRs of 93, 115 and 115 words raise nothing); a trial merge of this branch with each open pull request that changes the same files (no conflict added; with #1454, which merged first, the 51 tests passed on the merge of this branch with its head before the rebase). **Not tested:** how the forge's new pull request page shows the template (the headings, the stand-ins, the comments hidden): the page cannot be opened for a branch that is not on main yet; `forge.py pr create` against the new template (it opens a pull request: not run; its calls into the check were read, and their shapes did not change); the workflow itself on the forge (it runs main's copy, so it runs on the first pull request event after the merge); Windows (the check is plain Python text rules: not run); a `feat` or `perf` description made from the new template (the tests use a fix, a perf and a docs sample). **Must be tested:** - the forge's new pull request page, any browser, after merge: start a new pull request from any branch of a fork and look at the description box, expecting the status line, then the TL;DR and Breakdown headings with their stand-ins and no comment text showing (by hand: look at the box, and close it without creating anything) ## Risks **What could break:** a description that keeps a TL;DR over 120 words, a Breakdown without bullets or with an empty bullet, or a placeholder in either, is refused when it is marked ready, and the workflow flags it on its next edit. No open description is in that case now: two were when the check first read them (#1466's TL;DR was 191 words and #68's 155, both added today) and both were cut to 115 words; a looser limit is one constant, `TLDR_WORDS`. A `## TL;DR` or `## Breakdown` heading used for something else is read as these; none of the 337 descriptions read at the first look had one. **Cost:** the check reads two short sections more: 0.28 to 0.29 s against 0.27 to 0.29 s over 335 descriptions (about 0.8 ms each), on a loaded PC, so no measurable difference. **Turning it off:** revert the commit, or delete the two sections from a description (it then only warns). **Compatibility:** nothing changes for a player; a contributor who starts a pull request from the new template sees two more sections to fill in, and a description from the old template passes with two warnings. ## Overlaps Read from `pulls/N/files` of the open pull requests and trial merges (`git merge-tree --write-tree`) of each head against main and against this branch, conflict hunks read where there were any (done on main `dcb7b47c2`; the commits added to main since touch none of this branch's files). #1454, which changed the same template, check, tests, good.md and CONTRIBUTING.md, merged while this was written: this branch was rebased onto it (the range-diff of the two commits shows only changed context lines) and the tests and the suite were rerun on the result. Fourteen open pull requests change CONTRIBUTING.md and nothing else of this branch's files (#29, #56, #67, #68, #144, #204, #235, #236, #284, #405, #685, #860, #893, #1145): eight merge cleanly with main and with this branch; six conflict with main already (#29 in CONTRIBUTING.md and tests/run_all.py, #204, #236, #284, #405, #685 in other files) and this branch adds no conflict to any of them. #768's file list was not readable (`null`); its head merges cleanly with both. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named): 38 tests fail on both main and this PR, the same names, none in the description check: test_bug_report_defaults.py, test_cpu_report.py, test_crt_options.py, test_damage_debug_menu.py, test_display_startup_failure.py, test_dlss_feature_size.py, test_failed_compile_keeps_objects.py, test_framegen_preset.py, test_fsr_uniform_ownership.py, test_guest_imported.py, test_hud_layout_row.py, test_import_reach.py, test_invalidate_vcore.py, test_linux_preflight.py, test_linux_pressure.py, test_linux_update.py, test_log_open_failure.py, test_mirror_budget.py, test_missing_nup.py, test_mods_size_table.py, test_nonuniform_image_arrays.py, test_parallel_init_order.py, test_pending_guest_import.py, test_pipeline_play_cache.py, test_present_thread.py, test_profiling_mode.py, test_readback_timeout_source.py, test_replay_write_order.py, test_rumble_menu.py, test_run_all_changed.py, test_slow_frames.py, test_staging_burst.py, test_target_image_pool.py, test_upscaler_round_trip.py, test_upscaler_rows.py, test_vcore_escape.py, test_vk_occlusion_queries.py, test_walk_key.py. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened (`tests/test_check_pr_description.py` reads the template, the check and CONTRIBUTING.md; its samples and `test_the_sections_follow_the_order`, `test_without_reproduce` and `test_refusals` follow the new sections, and nothing was removed). - [x] Docs updated. New on this forge; not opened on GitHub.
A description that keeps a TL;DR or a Breakdown after the status line must fill them in: no placeholder, the TL;DR in
120 words or fewer, the Breakdown with bullets and none of them empty. One that leaves them out still passes, with a
warning that suggests them, so the open pull requests stay valid (the tests take the good sample without the two
sections for one of those). The status line still comes first, and the template has both sections right after it.
tests/pr_description/good.md, the sample of a good description, has both now.

These tests fail on the checker and the template as they are on main.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
docs(forge): every pull request description starts with a plain TL;DR and a Breakdown; the check suggests them where they are missing
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 11s
CI / Workflow lint (actionlint) (pull_request) Successful in 12s
Third-party parts / As their notes say (pull_request) Successful in 16s
PR description / Description follows the template (pull_request_target) Successful in 3s
Website / Website on Cloudflare (pull_request_target) Has been skipped
a51766cb99
A description is long, and a reader who is not a developer has to dig through it to learn what a pull request is
for. The template now has two sections right after the status line: a TL;DR (two to four plain sentences a player
could follow, 120 words at most: what was wrong or missing, what this changes, the headline result and how it was
checked) and a Breakdown (a short bullet each for who it helps, what was wrong, what changes, what does not change,
how it was checked, what is still open and the risk), each with a comment that explains it and a short example.

scripts/check_pr_description.py reads them. Kept, they are filled in: no placeholder left, the TL;DR in 120 words or
fewer, the Breakdown with bullets and none of them empty. Left out, they are a warning that suggests them and never an
error, so the open pull requests, written before the template had them, keep passing. The status line is still the
first line. CONTRIBUTING.md's "Commits and pull requests" lists the two sections.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
mooki changed title from WIP: docs(forge): every pull request description starts with a plain TL;DR and a Breakdown; the check suggests them where they are missing to docs(forge): every pull request description starts with a plain TL;DR and a Breakdown; the check suggests them where they are missing 2026-11-17 21:05:11 +00:00
Author
Member

The script behind the description's scan of the open pull requests is attached: scan_open_prs.py (attached as scan_open_prs.py.txt; the forge does not take .py files). It reads every open pull request's description through the forge's public API, page by page until an empty page, and runs main's description check and this branch's on each one, with the pull request's own diff and title; it prints the errors each check adds or drops and the warnings. It only reads: it posts nothing and changes nothing. To run it, put it in a checkout of this branch with main's scripts/check_pr_description.py saved beside it as the old check, and run it with Python 3.12 or newer.

The script behind the description's scan of the open pull requests is attached: [`scan_open_prs.py`](https://git.magicbean.dev/attachments/98776827-f09d-4ce3-9c83-a1a7751349d0) (attached as `scan_open_prs.py.txt`; the forge does not take `.py` files). It reads every open pull request's description through the forge's public API, page by page until an empty page, and runs main's description check and this branch's on each one, with the pull request's own diff and title; it prints the errors each check adds or drops and the warnings. It only reads: it posts nothing and changes nothing. To run it, put it in a checkout of this branch with main's `scripts/check_pr_description.py` saved beside it as the old check, and run it with Python 3.12 or newer.
mooki scheduled this pull request to auto merge when all checks succeed 2026-11-17 21:13:21 +00:00
llun approved these changes 2026-11-17 21:17:43 +00:00
mooki merged commit e8e92f97d8 into main 2026-11-17 21:17:47 +00:00
mooki deleted branch docs/pr-template-tldr 2026-11-17 21:17:47 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1470
No description provided.