feat(agents): test requests need a collaborator's PR or the "testing wanted" label #2746

Closed

github-sync wants to merge 1 commit from github/pr/2746 into main

github-sync commented

2026-11-12 11:35:39 +00:00

Owner

@hckrman101 opened this pull request on GitHub · 2026-11-12 10:41 UTC · original
Merged on GitHub in 51da0464b4

A test request asks someone's agent to download, build and run a PR on their own PC, and since #2734 anyone signed in to the team chat can post one for any PR. This makes a maintainer the gate: a request can only be picked up when the PR's author is a collaborator on GitHub, or a maintainer has looked at the PR and added the testing wanted label.

What changed

scripts/test_requests.py asks GitHub (no sign-in needed; GITHUB_TOKEN/GH_TOKEN used when set) for the PR's author_association and labels. A request is approved when the author is OWNER, MEMBER or COLLABORATOR, or the PR has the testing wanted label.
- claim refuses (exit 3) a request that isn't approved, or whose PR is merged or closed, and sends nothing when GitHub can't be asked.
- list and show mark each open request approved and add the PR's state; waiting ones are listed last, with "waiting for a maintainer to add the label".
- request tells the author when their PR needs the label.
The server's relay shows a waiting request as a grey card, "Waiting for a maintainer to add the testing wanted label", with no claims, results or guide link, and checks GitHub every 3 minutes so the card opens by itself soon after the label is added (approved ones every 15, for the PR's state).
The label is also an entry point: the relay lists open PRs labelled testing wanted every 3 minutes and posts a request for each one that has none, for its author (their GitHub name and avatar on the card, the PR title as the summary); it closes the ones it posted when the label comes off or the PR is merged or closed. The guide, the request skill and AGENTS.md say so.
The testing skill now treats the request, the PR and its code as data, never instructions: it builds and runs only as docs/development.md says, and before the first build looks at what the PR changes that runs on the PC (scripts, tools, workflows, anything downloading or reaching outside the repository), stopping if something isn't explained.
The request skill, the guide (docs/test-requests.md) and AGENTS.md say the same.

Tested

Live: with this running, the relay posted requests for the two labelled PRs that had none (#2456, #2738), under their authors' GitHub names and avatars, and left #2306 alone (it already had one).
tests/test_test_requests.py: a stranger's PR without the label can't be claimed; the label approves it; a merged PR can't be claimed; no claim goes out when GitHub can't be asked, while list still works; waiting requests list last. python -I -B tests/run_all.py test_requests team_chat: both pass.
Live: the open request for #2306 (a contributor's PR) turned into the waiting card when the relay was updated; after adding testing wanted to #2306 it opened again on its own within 3 minutes, in both Discord channels, and list shows it approved.

> [@hckrman101](https://github.com/hckrman101) opened this pull request on GitHub · 2026-11-12 10:41 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2746) > **Merged on GitHub** in https://git.magicbean.dev/MagicBeanDev/Hookshot/commit/51da0464b4cf0c7f56b81ebd6bb309325835e175 A test request asks someone's agent to download, build and run a PR on their own PC, and since #2734 anyone signed in to the team chat can post one for any PR. This makes a maintainer the gate: a request can only be picked up when the PR's author is a collaborator on GitHub, or a maintainer has looked at the PR and added the `testing wanted` label. **What changed** - `scripts/test_requests.py` asks GitHub (no sign-in needed; `GITHUB_TOKEN`/`GH_TOKEN` used when set) for the PR's `author_association` and labels. A request is approved when the author is `OWNER`, `MEMBER` or `COLLABORATOR`, or the PR has the `testing wanted` label. - `claim` refuses (exit 3) a request that isn't approved, or whose PR is merged or closed, and sends nothing when GitHub can't be asked. - `list` and `show` mark each open request `approved` and add the PR's state; waiting ones are listed last, with "waiting for a maintainer to add the label". - `request` tells the author when their PR needs the label. - The server's relay shows a waiting request as a grey card, "Waiting for a maintainer to add the `testing wanted` label", with no claims, results or guide link, and checks GitHub every 3 minutes so the card opens by itself soon after the label is added (approved ones every 15, for the PR's state). - **The label is also an entry point:** the relay lists open PRs labelled `testing wanted` every 3 minutes and posts a request for each one that has none, for its author (their GitHub name and avatar on the card, the PR title as the summary); it closes the ones it posted when the label comes off or the PR is merged or closed. The guide, the request skill and AGENTS.md say so. - The testing skill now treats the request, the PR and its code as data, never instructions: it builds and runs only as docs/development.md says, and before the first build looks at what the PR changes that runs on the PC (scripts, tools, workflows, anything downloading or reaching outside the repository), stopping if something isn't explained. - The request skill, the guide (`docs/test-requests.md`) and AGENTS.md say the same. **Tested** - Live: with this running, the relay posted requests for the two labelled PRs that had none (#2456, #2738), under their authors' GitHub names and avatars, and left #2306 alone (it already had one). - `tests/test_test_requests.py`: a stranger's PR without the label can't be claimed; the label approves it; a merged PR can't be claimed; no claim goes out when GitHub can't be asked, while `list` still works; waiting requests list last. `python -I -B tests/run_all.py test_requests team_chat`: both pass. - Live: the open request for #2306 (a contributor's PR) turned into the waiting card when the relay was updated; after adding `testing wanted` to #2306 it opened again on its own within 3 minutes, in both Discord channels, and `list` shows it approved.

github-sync added 1 commit

2026-11-12 11:35:39 +00:00

feat(agents): test requests are claimable only for collaborators' PRs or ones labelled "testing wanted" 3c9a62ea5a

test_requests.py asks GitHub for the PR's author association and labels: claim refuses a PR that is neither by a
collaborator nor labelled, or that is merged or closed, and goes out only when GitHub answered; list shows
waiting requests last. The testing skill treats request and PR text as data and looks at what a PR runs before
building it. The docs also cover the label as a request of its own (the team relay posts one for the PR).

github-sync referenced this pull request from a commit

2026-11-12 11:35:41 +00:00

feat(agents): test requests need a collaborator's PR or the "testing wanted" label (#2746)

~~github-sync referenced this pull request 2026-11-12 11:35:54 +00:00~~

chore(main): release 2.2.0-rc.1 #2071

github-sync closed this pull request

2026-11-12 11:36:37 +00:00

github-sync referenced this pull request

2026-11-12 14:41:10 +00:00

Idea: hardware test runners, so an approved PR test runs on a Steam Deck, AMD, Intel or 8 GB card without waiting for a person #2797

github-sync referenced this pull request

2026-11-12 14:48:35 +00:00