feat: semi-live logs of running jobs #77

Merged
HumanBean merged 1 commit from HumanBean/Beanstalk:feat/live-logs into main 2026-11-16 21:13:49 +00:00
Owner

Why

A job that seemed stuck (#1152 on AstraLol's PC: at build-pr for 45+ minutes, where its PR builds usually take 2-3) could not be looked into from the farm. Nothing of the tester's log reached the farm until the report.

What changed

  • Agent: LiveLog collects the job's log lines (the agent's own, plus the build's and the game's, as agent.log has them). Each heartbeat (every few seconds) carries the newest ones in log, at most 100 lines of 300 characters, scrubbed as a report is (Scrubber.ForThisPc). It holds at most 1000 lines; if heartbeats fail for a while, the oldest go and a line says how many.
  • Coordinator: scheduler.log_lines checks them and append_log adds them to jobs.live_log while that lease holds, keeping the last 64 KB, cut at a line. Migration 0018 adds jobs.live_log and live_log_at. The log is emptied on a new lease and kept after the job ends.
  • Dashboard: /jobs/<id>/log shows the lines and reloads every 15 s while the job runs. PR and run pages link it from each job that has started.
  • Docs: protocol.md (the heartbeat's log).

Tests

  • Coordinator: test_heartbeat_lines_make_the_live_log (append, another agent's lines refused, the 64 KB cut, reset on a new lease, bad bodies). The e2e flow sends a heartbeat with a line, reads the page (escaped, reloading) and the PR page's link, and gets 400 and 404 where it should. Suite: 541 passed, 1 skipped.
  • Agent: LiveLog checks (scrubbed, cut, each line once, a pile-up). 1959 checks passed; the release build has no warnings.
## Why A job that seemed stuck (#1152 on AstraLol's PC: at build-pr for 45+ minutes, where its PR builds usually take 2-3) could not be looked into from the farm. Nothing of the tester's log reached the farm until the report. ## What changed - **Agent:** `LiveLog` collects the job's log lines (the agent's own, plus the build's and the game's, as `agent.log` has them). Each heartbeat (every few seconds) carries the newest ones in `log`, at most 100 lines of 300 characters, scrubbed as a report is (`Scrubber.ForThisPc`). It holds at most 1000 lines; if heartbeats fail for a while, the oldest go and a line says how many. - **Coordinator:** `scheduler.log_lines` checks them and `append_log` adds them to `jobs.live_log` while that lease holds, keeping the last 64 KB, cut at a line. Migration 0018 adds `jobs.live_log` and `live_log_at`. The log is emptied on a new lease and kept after the job ends. - **Dashboard:** `/jobs/<id>/log` shows the lines and reloads every 15 s while the job runs. PR and run pages link it from each job that has started. - **Docs:** protocol.md (the heartbeat's `log`). ## Tests - Coordinator: `test_heartbeat_lines_make_the_live_log` (append, another agent's lines refused, the 64 KB cut, reset on a new lease, bad bodies). The e2e flow sends a heartbeat with a line, reads the page (escaped, reloading) and the PR page's link, and gets 400 and 404 where it should. Suite: 541 passed, 1 skipped. - Agent: LiveLog checks (scrubbed, cut, each line once, a pile-up). 1959 checks passed; the release build has no warnings.
feat: semi-live logs of running jobs
Some checks are pending
CI / Coordinator tests (pull_request) Waiting to run
CI / Agent build and checks (pull_request) Waiting to run
CI / Coordinator image (pull_request) Waiting to run
CI / Conventional Commit title (pull_request) Successful in 6s
CI / Workflow lint (actionlint) (pull_request) Successful in 7s
e403a9d56a
A job that seemed stuck could not be looked into from the farm: nothing of
the tester's log reached it until the report. Each heartbeat (every few
seconds) now carries the job's newest log lines (the agent's own and the
build's and game's, as agent.log has them), at most 100 of 300 characters,
scrubbed as a report is. The coordinator keeps the last 64 KB per job
(migration 0018: jobs.live_log, live_log_at; emptied on a new lease), and
/jobs/<id>/log shows them, reloading every 15 s while the job runs; the PR
and run pages link it from each job.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
HumanBean deleted branch feat/live-logs 2026-11-16 21:13:49 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Beanstalk!77
No description provided.