docs: the Test Farm page, which replaces test requests #542

Open

HumanBean wants to merge 1 commit from HumanBean/Longshot:docs/test-farm into main

pull from: HumanBean/Longshot:docs/test-farm

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 2 Commits 1 Files changed 16 +176 -1087

HumanBean commented 2026-11-14 18:54:08 +00:00

Owner

• Copy link
• Copy Markdown

Why

Volunteers can now lend their PCs to test pull requests through the Test Farm (Beanstalk), but nothing on the site says what it is, whether it's safe, or how to join. Maintainers and PR authors also need the /beanstalk and @Beanstalk commands written down.

The farm replaces test requests (the #test-requests room, scripts/test_requests.py, /testing and /request-testing): a PR that needs another PC's hardware now asks @Beanstalk, so the old system is removed rather than kept beside it.

What changed

• docs: new docs/test-farm.md:
  • what the farm is;
  • why it is safe and where it isn't (per-commit approval, two maintainers for build paths, data-only plans, the sandbox, the plain statement about GPU drivers);
  • what leaves the PC and what never does;
  • requirements, setup (app and CLI) and the policy file;
  • testing one PR locally;
  • for PR authors: /beanstalk, @Beanstalk <words>, /beanstalk plan, the 👀 reaction, AI-written plans, statuses and the results comment.
• website: published as contributing/test-farm (website/src/content-pages.ts), with a sidebar entry "Test Farm" under Development (website/astro.config.ts).
• docs: docs/reporting-and-contributing.md links to it from "Help with testing".
• removed: test requests. scripts/test_requests.py and tests/test_test_requests.py, docs/test-requests.md, the .claude/skills/testing and .claude/skills/request-testing skills.
• AGENTS.md: "Test requests" becomes "Testing on other hardware" (ask @Beanstalk; changes that need a person's eyes say where and how in Testing and stay WIP).
• Mentions of test requests dropped from CONTRIBUTING.md, docs/forge.md, docs/workflow.md (the room's row), scripts/forge.py and scripts/magicbean_login.py docstrings, and the team page.

Flags and settings

None.

Measurements

Not applicable: documentation only, no code or performance change.

main: 7ff5256e (2026-11-14)
this PR: docs/test-farm @ b43d8529

	main @ 7ff5256e	this PR @ b43d8529
site build and npm run check	53 checks pass	53 checks pass, plus the new page

Testing

Tested: cd website && npm ci && npm run build && npm run check passes: 53 checks, including the brand-word guard. The built page /contributing/test-farm/ renders and its links resolve: Beanstalk's repository, spec, releases and docs/test-plan.md; Longshot's /requirements/ and CONTRIBUTING.md (that site run was on 43d71eb1, before the removal). After the removal: tests/test_forge_pr.py and tests/test_team_chat.py pass, and every relative .md link in the repo resolves except CONTRIBUTING.md's PR-description line, which the PR-template PR rewrites.

Not tested: the website build after the removal (only TeamRoster.astro's sentence and docs changed). The Beanstalk releases link points at a page with no release yet. The page says so and gives build-from-source steps meanwhile.

Needs other testers: None.

Screenshots

Nothing on screen beyond a docs page.

Risk and rollback

Docs, plus removing the test-request script and skills; reverting the commit brings them back. The #test-requests Matrix room and its Discord relay (magicbean-infra) are not touched here.

Reviewer notes

Merge after the PR-description PR, which takes test requests out of the template, the checker and forge.py pr ready; until then those still point at the removed script. The page describes targets by tag (steamdeck, radv), which Beanstalk's next coordinator update ships. The commands and behaviour described match Beanstalk's PR (MagicBeanDev/Beanstalk#3), whose coordinator is live at https://beanstalk.magicbean.dev. Wording follows the existing help pages.

Checklist

For everyone

• One topic.
• The title is a Conventional Commit (type(scope): description).
• Rebased on current main.
• python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above). This change is docs only; the earlier run on main plus this change had the same failures as main (test_bake_portability, test_program_cache_cleanup, test_control and test_longshot_ctl in the sandbox, test_pad_layout, test_program_cache_one_variant, test_vram_budget_row).
• Tested in the game, or what wasn't is named under Testing.
• Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
• Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. Not applicable (docs only).
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• Docs updated.

## Why Volunteers can now lend their PCs to test pull requests through the Test Farm ([Beanstalk](https://git.magicbean.dev/MagicBeanDev/Beanstalk)), but nothing on the site says what it is, whether it's safe, or how to join. Maintainers and PR authors also need the `/beanstalk` and `@Beanstalk` commands written down. The farm replaces test requests (the `#test-requests` room, `scripts/test_requests.py`, `/testing` and `/request-testing`): a PR that needs another PC's hardware now asks `@Beanstalk`, so the old system is removed rather than kept beside it. ## What changed - docs: new `docs/test-farm.md`: - what the farm is; - why it is safe and where it isn't (per-commit approval, two maintainers for build paths, data-only plans, the sandbox, the plain statement about GPU drivers); - what leaves the PC and what never does; - requirements, setup (app and CLI) and the policy file; - testing one PR locally; - for PR authors: `/beanstalk`, `@Beanstalk <words>`, `/beanstalk plan`, the 👀 reaction, AI-written plans, statuses and the results comment. - website: published as `contributing/test-farm` (`website/src/content-pages.ts`), with a sidebar entry "Test Farm" under Development (`website/astro.config.ts`). - docs: `docs/reporting-and-contributing.md` links to it from "Help with testing". - removed: test requests. `scripts/test_requests.py` and `tests/test_test_requests.py`, `docs/test-requests.md`, the `.claude/skills/testing` and `.claude/skills/request-testing` skills. - AGENTS.md: "Test requests" becomes "Testing on other hardware" (ask `@Beanstalk`; changes that need a person's eyes say where and how in Testing and stay WIP). - Mentions of test requests dropped from CONTRIBUTING.md, `docs/forge.md`, `docs/workflow.md` (the room's row), `scripts/forge.py` and `scripts/magicbean_login.py` docstrings, and the team page. ## Flags and settings None. ## Measurements Not applicable: documentation only, no code or performance change. main: 7ff5256e (2026-11-14) this PR: docs/test-farm @ b43d8529 | | main @ 7ff5256e | this PR @ b43d8529 | |---|---|---| | site build and `npm run check` | 53 checks pass | 53 checks pass, plus the new page | ## Testing **Tested:** `cd website && npm ci && npm run build && npm run check` passes: 53 checks, including the brand-word guard. The built page `/contributing/test-farm/` renders and its links resolve: Beanstalk's repository, spec, releases and `docs/test-plan.md`; Longshot's `/requirements/` and CONTRIBUTING.md (that site run was on 43d71eb1, before the removal). After the removal: `tests/test_forge_pr.py` and `tests/test_team_chat.py` pass, and every relative `.md` link in the repo resolves except CONTRIBUTING.md's PR-description line, which the PR-template PR rewrites. **Not tested:** the website build after the removal (only `TeamRoster.astro`'s sentence and docs changed). The Beanstalk releases link points at a page with no release yet. The page says so and gives build-from-source steps meanwhile. **Needs other testers:** None. ## Screenshots Nothing on screen beyond a docs page. ## Risk and rollback Docs, plus removing the test-request script and skills; reverting the commit brings them back. The `#test-requests` Matrix room and its Discord relay (magicbean-infra) are not touched here. ## Reviewer notes **Merge after the PR-description PR**, which takes test requests out of the template, the checker and `forge.py pr ready`; until then those still point at the removed script. The page describes targets by tag (`steamdeck`, `radv`), which Beanstalk's next coordinator update ships. The commands and behaviour described match Beanstalk's PR (MagicBeanDev/Beanstalk#3), whose coordinator is live at https://beanstalk.magicbean.dev. Wording follows the existing help pages. ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). This change is docs only; the earlier run on main plus this change had the same failures as main (test_bake_portability, test_program_cache_cleanup, test_control and test_longshot_ctl in the sandbox, test_pad_layout, test_program_cache_one_variant, test_vram_budget_row). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. Not applicable (docs only). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Docs updated.

HumanBean added 1 commit 2026-11-14 18:54:09 +00:00

docs: how to help test with the Test Farm (Beanstalk) ... a43e82a860

A page on lending a PC to the Test Farm: what it is, why it is safe and
where it is not, what leaves the PC, setting up the agent, the policy
file, testing one PR locally, and for PR authors the /beanstalk and
@Beanstalk commands and the AI-written plans. Published as
contributing/test-farm and linked from the reporting page.

Assisted-by: Claude Code
Co-Authored-By: Claude Opus 5.5 <[email protected]>

HumanBean changed title from WIP: docs: how to help test with the Test Farm (Beanstalk) to docs: how to help test with the Test Farm (Beanstalk) 2026-11-14 18:54:24 +00:00

HumanBean force-pushed docs/test-farm from a43e82a860 to 43d71eb15f 2026-11-14 19:28:02 +00:00 Compare

HumanBean commented 2026-11-14 19:28:02 +00:00

Author

Owner

• Copy link
• Copy Markdown

Pushed: the commands table follows Beanstalk's changes. Only BigBeans start the farm (on any PR, their own too); anyone can ask and a BigBean approves with 👍; run is folded into @Beanstalk; targets is gone.

Pushed: the commands table follows Beanstalk's changes. Only BigBeans start the farm (on any PR, their own too); anyone can ask and a BigBean approves with 👍; `run` is folded into `@Beanstalk`; `targets` is gone.

👀 1

Beanstalk commented 2026-11-14 19:28:04 +00:00

Member

• Copy link
• Copy Markdown

@HumanBean the PR got a new push (43d71eb) after your comment. Review it and approve again.

@HumanBean the PR got a new push (`43d71eb`) after your comment. Review it and approve again.

HumanBean force-pushed docs/test-farm from 43d71eb15f to b43d852908 2026-11-14 20:08:12 +00:00 Compare

HumanBean changed title from docs: how to help test with the Test Farm (Beanstalk) to docs: the Test Farm page, which replaces test requests 2026-11-14 20:08:39 +00:00

HumanBean force-pushed docs/test-farm from b43d852908 to 9d4e4515fe 2026-11-15 04:36:31 +00:00 Compare

mooki added the

kind

docs

platform/linux

platform/windows

priority

low

labels 2026-11-15 13:55:24 +00:00

mooki referenced this pull request 2026-11-15 18:42:19 +00:00

feat(test): pin test requests and results to commits #71

mooki referenced this pull request 2026-11-15 18:42:23 +00:00

ci: add an owner-approved hardware fixture pilot #100

Some checks failed

Hide checks

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Failing after 16s

Details

CI / Conventional Commit title (pull_request) Successful in 13s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 19s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 18s

Details

Third-party parts / As their notes say (pull_request) Successful in 24s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

This pull request has changes conflicting with the target branch.

• AGENTS.md
• scripts/test_requests.py
• website/astro.config.ts

View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Merge commit title docs: the Test Farm page, which replaces test requests (#542)

Merge commit body Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/542

docs: the Test Farm page, which replaces test requests (#542) Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/542 Copy merge commit message

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'docs/test-farm:HumanBean-docs/test-farm'

git switch 'HumanBean-docs/test-farm'

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/tooling

Developer tools: scripts, CI, build and release tooling

  

area/upscaling

DLSS, FSR, frame generation

  

beanstalk: approved @ 1697603

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 2c5a62e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 4fd82b7

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 64d6d09

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 6f91fae

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 70a78f9

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8015c9a

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8d361dc

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 9d8356e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ abf2227

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ d82f202

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ e5f1e40

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ ee8ab5f

Set by Beanstalk for show; the farm's database is what counts

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Affects Linux (a change for every system carries both Windows and Linux)

  

platform/macos

macOS (Apple silicon or Intel Macs)

  

platform/steam-deck

Steam Deck

  

platform/windows

Affects Windows (a change for every system carries both Windows and Linux)

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

regression

Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

ci: approved

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/tooling

area/upscaling

beanstalk: approved @ 1697603

beanstalk: approved @ 2c5a62e

beanstalk: approved @ 4fd82b7

beanstalk: approved @ 64d6d09

beanstalk: approved @ 6f91fae

beanstalk: approved @ 70a78f9

beanstalk: approved @ 8015c9a

beanstalk: approved @ 8d361dc

beanstalk: approved @ 9d8356e

beanstalk: approved @ abf2227

beanstalk: approved @ d82f202

beanstalk: approved @ e5f1e40

beanstalk: approved @ ee8ab5f

ci: approved

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/macos

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

regression

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!542

No description provided.