feat(coordinator): testers' details are for BigBeans; everyone else sees the testers in numbers #120

Merged
HumanBean merged 1 commit from HumanBean/Beanstalk:feat/tester-privacy into main 2026-11-20 19:09:06 +00:00
Owner

Summary

The Testers page (/agents) showed every tester's login, PC name, GPU model, driver, status and history to anyone. Now:

  • A signed-in BigBean sees it all as before: /agents, the health page's tester table and its "Needs a look" notes that name testers, and any tester's page.
  • A tester signed in sees their own page (/agents/<id>).
  • Everyone else sees the testers in numbers (web.tester_counts): registered, seen this week and online, what the online ones are doing (testing, ready, waiting, paused, a problem), and how many have each OS, GPU maker and kind of PC (dGPU, iGPU, Steam Deck, handheld), online and this week. No login, PC name, GPU model or driver. Hidden testers count, as they always did. A tester's page answers 403 with a page that says who may see it and offers sign-in. The health page's notes become counts ("3 testers are on an app older than nightly-204").

Hidden testers: a BigBean sees them too, marked hidden, on /agents, the health page and their own page (with their Discord name). For everyone else they stay out of every page, as before, and PR pages and reports say "Hidden tester" for all.

PR pages and reports still name the tester who ran each job, as before. The How page no longer says build changes need two approvals (#119).

Screenshots

The dev server's made-up data (beanstalk.devserver), on a throwaway database.

/agents signed out, desktop: the testers in numbers

/agents signed out, 390 px

/health signed out: counts instead of the tester table

/agents as a BigBean, desktop: every tester, as before

/agents as a BigBean, 390 px

Testing

Tested: the coordinator suite in a python:3.13-alpine container with Postgres 17 on the farm's server: 593 passed, 3 failed, 1 skipped. The 3 (test_regression.py) fail the same way on main. New test_tester_privacy.py: anyone sees no login, name, GPU or link on /agents and /health, with the counts (the hidden tester counted); the health page counts every registered tester and a PC with two setup warnings once; a hidden tester sees their own page; a tester page is 403 for anyone, 200 for its own tester (who still sees counts on the list) and for a BigBean, who sees both tables. Tests that check tester details now view as a BigBean (bigbean_view).

Still to test: after deploy, /agents signed out, signed in as a non-BigBean and as a BigBean.

🤖 Generated with Claude Code

## Summary The Testers page (`/agents`) showed every tester's login, PC name, GPU model, driver, status and history to anyone. Now: - **A signed-in BigBean** sees it all as before: `/agents`, the health page's tester table and its "Needs a look" notes that name testers, and any tester's page. - **A tester** signed in sees their own page (`/agents/<id>`). - **Everyone else** sees the testers in numbers (`web.tester_counts`): registered, seen this week and online, what the online ones are doing (testing, ready, waiting, paused, a problem), and how many have each OS, GPU maker and kind of PC (dGPU, iGPU, Steam Deck, handheld), online and this week. No login, PC name, GPU model or driver. Hidden testers count, as they always did. A tester's page answers 403 with a page that says who may see it and offers sign-in. The health page's notes become counts ("3 testers are on an app older than nightly-204"). **Hidden testers:** a BigBean sees them too, marked `hidden`, on `/agents`, the health page and their own page (with their Discord name). For everyone else they stay out of every page, as before, and PR pages and reports say "Hidden tester" for all. PR pages and reports still name the tester who ran each job, as before. The How page no longer says build changes need two approvals (#119). ## Screenshots The dev server's made-up data (beanstalk.devserver), on a throwaway database. ![/agents signed out, desktop: the testers in numbers](https://git.magicbean.dev/attachments/b2f87a08-389c-4835-82bf-ef4ee8016df8) ![/agents signed out, 390 px](https://git.magicbean.dev/attachments/0078657b-b6e5-406c-967a-c4252d248dd4) ![/health signed out: counts instead of the tester table](https://git.magicbean.dev/attachments/5a319b5f-4965-4833-b770-3a0ae747184e) ![/agents as a BigBean, desktop: every tester, as before](https://git.magicbean.dev/attachments/8a8280fb-e6fc-4c8a-9dbb-17d5fc01f495) ![/agents as a BigBean, 390 px](https://git.magicbean.dev/attachments/3e135a57-a3da-407d-a600-ad25a5bad7e5) ## Testing **Tested:** the coordinator suite in a python:3.13-alpine container with Postgres 17 on the farm's server: 593 passed, 3 failed, 1 skipped. The 3 (`test_regression.py`) fail the same way on main. New `test_tester_privacy.py`: anyone sees no login, name, GPU or link on `/agents` and `/health`, with the counts (the hidden tester counted); the health page counts every registered tester and a PC with two setup warnings once; a hidden tester sees their own page; a tester page is 403 for anyone, 200 for its own tester (who still sees counts on the list) and for a BigBean, who sees both tables. Tests that check tester details now view as a BigBean (`bigbean_view`). **Still to test:** after deploy, `/agents` signed out, signed in as a non-BigBean and as a BigBean. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
feat(coordinator): testers' details are for BigBeans; everyone else sees the testers in numbers
Some checks failed
CI / Conventional Commit title (pull_request) Successful in 19s
CI / Workflow lint (actionlint) (pull_request) Successful in 20s
CI / Coordinator tests (pull_request) Has been cancelled
CI / Agent build and checks (pull_request) Has been cancelled
CI / Coordinator image (pull_request) Has been cancelled
65a1bc334c
/agents, the health page's tester table and its notes that name testers now
show each tester (login, PC name, GPU model, driver, status, history) only to
a signed-in BigBean (web.sees_testers). Everyone else gets web.tester_counts:
registered, seen this week and online, what the online ones are doing, and how
many have each OS, GPU maker and kind of PC, online and this week; no login,
name, GPU model or driver. A tester's own page (/agents/<id>) is for BigBeans
and that tester; anyone else gets a 403 page that says so.

Assisted-by: Claude Code
Co-Authored-By: Claude Opus 5.5 <[email protected]>
BeanBot left a comment

Recommendation: Request changes. The new health counts misstate registered testers and setup warnings, and hidden testers cannot access their own pages despite the stated access policy. The visual change also lacks the screenshots required by the repository.

The role checks and aggregate view reduce direct exposure of tester details on the affected pages, but the aggregates must be accurate and the promised owner access must work for hidden testers. Add desktop and phone-width screenshots using fictional data.

Does it help: Unproven: the PR reports 590 passing tests, three failures also seen on main, and one skip; the new tests cover basic signed-out, owner and BigBean access, but do not exercise hidden owners, old registrations or multiple warnings per tester. No independent run was performed.
Other systems: The shared counts appear on both /agents and /health; health fetches a different population. Hidden testers are intentionally still included in aggregates. PR and report identities remain public as described. Authentication uses the existing Forgejo session and membership checks.
Switches: None added, changed or removed.

Reviewed: 65a1bc334c against d6e717d10d: Read the base AGENTS.md and CONTRIBUTING.md; no docs/review-checklist.md or .forgejo PR template was present. Read the diff, new templates, health template, privacy tests, conftest, health.py, agents.py and auth.py; web.py was truncated by the file tool and the remaining changes, other changed tests and documentation were reviewed from the diff only. No code was run.; nothing run (a code review).
Checked and fine:

  • The new counts use an allowlist of hardware tags and include hidden testers.
  • The signed-in owner comparison is case-insensitive, and non-owner detail pages return 403 for visible testers.

Blocking (to change before merge)

  1. coordinator/beanstalk/web.py:439: /health passes only agents seen in the last seven days into tester_counts, so its Registered and reports-in-all totals omit older active registrations and disagree with /agents. Query all non-retired agents for these totals and retain the seven-day filter for weekly counts; test an agent last seen over a week ago.
  2. coordinator/beanstalk/web.py:436: setups contains one entry per warning, not per tester. A single online PC missing both a sandbox and game files is reported as two testers with a setup warning. Count distinct agent IDs and test a PC with multiple warnings.
  3. coordinator/beanstalk/web.py:754: the hidden check returns 404 before the owner authorization check. A signed-in tester who chose hidden cannot see their own page, contrary to the stated policy. Permit the authenticated owner while retaining 404 for other viewers, and test that case.
  4. coordinator/beanstalk/templates/agents_counts.html:5: this changes the visible dashboard but the PR has no screenshots. CONTRIBUTING.md requires before/after dashboard screenshots at desktop and phone width, made with fictional data; add them to the description.

Testing: Reported suite: 590 passed, 3 failed as on main, 1 skipped. Post-deploy role checks are still outstanding; this review did not execute tests.
Overlaps and merge order: No overlapping open PR found in searches for tester privacy, agents and health.


Automated code review by BeanBot (gpt-6.1-terra), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** The new health counts misstate registered testers and setup warnings, and hidden testers cannot access their own pages despite the stated access policy. The visual change also lacks the screenshots required by the repository. The role checks and aggregate view reduce direct exposure of tester details on the affected pages, but the aggregates must be accurate and the promised owner access must work for hidden testers. Add desktop and phone-width screenshots using fictional data. **Does it help:** Unproven: the PR reports 590 passing tests, three failures also seen on main, and one skip; the new tests cover basic signed-out, owner and BigBean access, but do not exercise hidden owners, old registrations or multiple warnings per tester. No independent run was performed. **Other systems:** The shared counts appear on both /agents and /health; health fetches a different population. Hidden testers are intentionally still included in aggregates. PR and report identities remain public as described. Authentication uses the existing Forgejo session and membership checks. **Switches:** None added, changed or removed. **Reviewed:** 65a1bc334c78 against d6e717d10d97: Read the base AGENTS.md and CONTRIBUTING.md; no docs/review-checklist.md or .forgejo PR template was present. Read the diff, new templates, health template, privacy tests, conftest, health.py, agents.py and auth.py; web.py was truncated by the file tool and the remaining changes, other changed tests and documentation were reviewed from the diff only. No code was run.; nothing run (a code review). **Checked and fine:** - The new counts use an allowlist of hardware tags and include hidden testers. - The signed-in owner comparison is case-insensitive, and non-owner detail pages return 403 for visible testers. **Blocking** (to change before merge) 1. coordinator/beanstalk/web.py:439: /health passes only agents seen in the last seven days into tester_counts, so its Registered and reports-in-all totals omit older active registrations and disagree with /agents. Query all non-retired agents for these totals and retain the seven-day filter for weekly counts; test an agent last seen over a week ago. 2. coordinator/beanstalk/web.py:436: setups contains one entry per warning, not per tester. A single online PC missing both a sandbox and game files is reported as two testers with a setup warning. Count distinct agent IDs and test a PC with multiple warnings. 3. coordinator/beanstalk/web.py:754: the hidden check returns 404 before the owner authorization check. A signed-in tester who chose hidden cannot see their own page, contrary to the stated policy. Permit the authenticated owner while retaining 404 for other viewers, and test that case. 4. coordinator/beanstalk/templates/agents_counts.html:5: this changes the visible dashboard but the PR has no screenshots. CONTRIBUTING.md requires before/after dashboard screenshots at desktop and phone width, made with fictional data; add them to the description. **Testing:** Reported suite: 590 passed, 3 failed as on main, 1 skipped. Post-deploy role checks are still outstanding; this review did not execute tests. **Overlaps and merge order:** No overlapping open PR found in searches for tester privacy, agents and health. --- <sub>Automated code review by BeanBot (gpt-6.1-terra), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -0,0 +2,4 @@
{% from "_tester_counts.html" import tester_counts with context %}
{% block title %}Testers{% endblock %}
{% block main %}
<header class="page-head">
Member

Please add the required before/after desktop and phone-width dashboard screenshots to the PR description, using fictional tester data.

Please add the required before/after desktop and phone-width dashboard screenshots to the PR description, using fictional tester data.
@ -433,0 +433,4 @@
full = sees_testers(request)
if not full:
s["tester_rows"], s["hidden_testers"] = [], 0
s["setups_n"], s["old_n"] = len(s.get("setups") or []), len((s.get("testers") or {}).get("old") or [])
Member

setups has one entry per warning, not per tester. One PC missing both sandbox and game files is reported as two testers; count distinct agent IDs instead and cover multiple warnings on one PC.

`setups` has one entry per warning, not per tester. One PC missing both sandbox and game files is reported as two testers; count distinct agent IDs instead and cover multiple warnings on one PC.
@ -433,0 +436,4 @@
s["setups_n"], s["old_n"] = len(s.get("setups") or []), len((s.get("testers") or {}).get("old") or [])
s["setups"] = []
s["testers"] = dict(s["testers"], old=[])
return page(request, "health.html", s=s, full=full, counts=None if full else tester_counts(rows))
Member

These rows were restricted to the last seven days, so Registered and reports in all on /health omit older active testers and disagree with /agents. Fetch all non-retired rows for these totals and test an older registration.

These rows were restricted to the last seven days, so Registered and reports in all on /health omit older active testers and disagree with /agents. Fetch all non-retired rows for these totals and test an older registration.
Member

The hidden check runs before owner authorization, so a hidden tester gets 404 on their own page, contrary to the stated owner access. Allow the signed-in owner while keeping it hidden from other viewers, with a regression test.

The hidden check runs before owner authorization, so a hidden tester gets 404 on their own page, contrary to the stated owner access. Allow the signed-in owner while keeping it hidden from other viewers, with a regression test.
HumanBean force-pushed feat/tester-privacy from 65a1bc334c
Some checks failed
CI / Conventional Commit title (pull_request) Successful in 19s
CI / Workflow lint (actionlint) (pull_request) Successful in 20s
CI / Coordinator tests (pull_request) Has been cancelled
CI / Agent build and checks (pull_request) Has been cancelled
CI / Coordinator image (pull_request) Has been cancelled
to b7bdc6cc50
Some checks failed
CI / Conventional Commit title (pull_request) Successful in 9s
CI / Workflow lint (actionlint) (pull_request) Successful in 9s
CI / Coordinator tests (pull_request) Has been cancelled
CI / Agent build and checks (pull_request) Has been cancelled
CI / Coordinator image (pull_request) Has been cancelled
2026-11-20 18:42:37 +00:00
Compare
HumanBean force-pushed feat/tester-privacy from b7bdc6cc50
Some checks failed
CI / Conventional Commit title (pull_request) Successful in 9s
CI / Workflow lint (actionlint) (pull_request) Successful in 9s
CI / Coordinator tests (pull_request) Has been cancelled
CI / Agent build and checks (pull_request) Has been cancelled
CI / Coordinator image (pull_request) Has been cancelled
to 8e93dc6cf7
Some checks failed
CI / Conventional Commit title (pull_request) Successful in 11s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
CI / Coordinator tests (pull_request) Has been cancelled
CI / Agent build and checks (pull_request) Has been cancelled
CI / Coordinator image (pull_request) Has been cancelled
2026-11-20 18:47:53 +00:00
Compare
HumanBean changed title from WIP: feat(coordinator): testers' details are for BigBeans; everyone else sees the testers in numbers to feat(coordinator): testers' details are for BigBeans; everyone else sees the testers in numbers 2026-11-20 18:47:54 +00:00
HumanBean force-pushed feat/tester-privacy from 8e93dc6cf7
Some checks failed
CI / Conventional Commit title (pull_request) Successful in 11s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
CI / Coordinator tests (pull_request) Has been cancelled
CI / Agent build and checks (pull_request) Has been cancelled
CI / Coordinator image (pull_request) Has been cancelled
to d0bb8d15b5
Some checks failed
CI / Conventional Commit title (pull_request) Successful in 17s
CI / Workflow lint (actionlint) (pull_request) Successful in 19s
CI / Coordinator tests (pull_request) Failing after 1m23s
CI / Agent build and checks (pull_request) Successful in 52s
CI / Coordinator image (pull_request) Successful in 13s
2026-11-20 18:55:16 +00:00
Compare
Author
Owner

Answered in d0bb8d15:

  1. The health page's counts now come from every non-retired tester (as /agents's); its table stays the last 7 days'. Tested with a tester last seen 10 days ago.
  2. Setup warnings and old apps are counted per tester (distinct agent ids). Tested with one PC missing both a sandbox and game files.
  3. Already fixed in 8e93dc6c (after the reviewed commit): a hidden tester signed in sees their own page; 404 for everyone else. Tested.
  4. Screenshots added to the description (desktop and 390 px, signed out and as a BigBean, the dev server's made-up data).
Answered in d0bb8d15: 1. The health page's counts now come from every non-retired tester (as /agents's); its table stays the last 7 days'. Tested with a tester last seen 10 days ago. 2. Setup warnings and old apps are counted per tester (distinct agent ids). Tested with one PC missing both a sandbox and game files. 3. Already fixed in 8e93dc6c (after the reviewed commit): a hidden tester signed in sees their own page; 404 for everyone else. Tested. 4. Screenshots added to the description (desktop and 390 px, signed out and as a BigBean, the dev server's made-up data).
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Beanstalk!120
No description provided.