fix: a job a tester lost goes to another tester first; a refused report says why #73

Merged
HumanBean merged 1 commit from HumanBean/Beanstalk:fix/lost-jobs-elsewhere into main 2026-11-16 17:18:40 +00:00
Owner

Why

  • The same PC kept taking back a job it lost: a lost lease or a passing failure requeued the job with no wait for that PC. Tommy's PC took #910 four times in a row, losing its lease each time, while other eligible PCs waited.
  • Leases ran out at the report stage, silently: the app was still polling (it took the job again 3 s after each expiry), so its report was refused or could not be sent. A refusal only went into the tester's own history, and the coordinator logged nothing.

What changed

  • Others go first: requeue_passing stamps agent_fails[<agent>].lost_at, and failed_out keeps that PC off the job for RETRY_WAIT (15 min). Any other eligible PC is offered it at once; if none takes it, the same PC tries again afterwards. Counted failures already waited.
  • Agent: when a report is refused (not a 409) or cannot be sent (too large), it sends fail with stage report and the reason. The attempt then closes with why, and the job requeues at once.
  • Coordinator: every refused report is logged (job, tester, status, reason).
  • Docs: protocol.md, architecture.md.

Tests

  • New test_a_job_one_pc_lost_goes_to_another_first. The lost-lease and passing-fail tests now check the wait.
  • Coordinator: 540 passed, 1 skipped. Agent: 1935 checks passed; release build has no warnings.
## Why - **The same PC kept taking back a job it lost:** a lost lease or a passing failure requeued the job with no wait for that PC. Tommy's PC took #910 four times in a row, losing its lease each time, while other eligible PCs waited. - **Leases ran out at the report stage, silently:** the app was still polling (it took the job again 3 s after each expiry), so its report was refused or could not be sent. A refusal only went into the tester's own history, and the coordinator logged nothing. ## What changed - **Others go first:** `requeue_passing` stamps `agent_fails[<agent>].lost_at`, and `failed_out` keeps that PC off the job for `RETRY_WAIT` (15 min). Any other eligible PC is offered it at once; if none takes it, the same PC tries again afterwards. Counted failures already waited. - **Agent:** when a report is refused (not a 409) or cannot be sent (too large), it sends `fail` with stage `report` and the reason. The attempt then closes with why, and the job requeues at once. - **Coordinator:** every refused report is logged (job, tester, status, reason). - **Docs:** protocol.md, architecture.md. ## Tests - New `test_a_job_one_pc_lost_goes_to_another_first`. The lost-lease and passing-fail tests now check the wait. - Coordinator: 540 passed, 1 skipped. Agent: 1935 checks passed; release build has no warnings.
fix: a job a tester lost goes to another tester first; a refused report says why
Some checks are pending
CI / Coordinator tests (pull_request) Waiting to run
CI / Agent build and checks (pull_request) Waiting to run
CI / Coordinator image (pull_request) Waiting to run
CI / Conventional Commit title (pull_request) Successful in 7s
CI / Workflow lint (actionlint) (pull_request) Successful in 6s
5a86bfc63f
A lost lease or a passing failure put the job straight back in the queue
with nothing to stop the same PC taking it again at once: Tommy's PC took
#910 four times in a row, losing its lease each time, while other Windows
PCs waited. Now that PC waits 15 minutes (agent_fails' lost_at, read by
failed_out), as after a counted failure, so any other PC that may take the
job is offered it first; if none does, the same PC tries again after that.

Those leases ran out at the report stage: the app was still polling (it
took the job again 3 s after each expiry), so its report never got in. A
refused report was only in the tester's own history and the job waited
for its lease to run out. The agent now sends fail with the refusal (or
the reason it could not be sent), and the coordinator logs every refused
report with its reason.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
HumanBean deleted branch fix/lost-jobs-elsewhere 2026-11-16 17:18:41 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Beanstalk!73
No description provided.