fix(agent): a teleport the game never carries out is sent again, MinGit comes before the self-test, and broken lifted C is built again #129
No reviewers
Labels
No labels
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
priority
critical
priority
high
priority
low
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Beanstalk!129
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Beanstalk:fix/agent-reliability"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Three app fixes for failures the farm keeps hitting:
\Device\Nullgrants AppContainers nothing (no ALL APPLICATION PACKAGES entry), so git dies at its firstopen("/dev/null"). The self-test's git check only ran once a job had fetched MinGit, so each such PC failed its first job, then left the AppContainer. The self-test now fetches MinGit first.code_*.cthat every other PC compiles. A compile error inwork/sa/gen/code_*.cnow builds the base once more from nothing.Tests: the agent, 2310 checks passed on a throwaway container. Nothing on screen changes.
🤖 Generated with Claude Code
- A teleport the game took but never carried out (still in play on the map it left 20 s later) is sent again, up to 3 times. 61 game starts in 3 days ran out of time at the wait for the map after it, on main and PRs alike, on Windows and Linux ("then the game was in play on KokiriForest_Child"). - The Windows self-test fetches the agent's MinGit before its git check. On some Windows builds \Device\Null grants AppContainers nothing, so git dies at its first open of /dev/null; the check ran only once a job had fetched MinGit, so each such PC failed its first job before it left the AppContainer. - A compile error in main's lifted C (work/sa/gen/code_*.c) in a kept work folder builds the base once more from nothing: one PC failed 5 jobs at a different line of a different file each time, code every other PC compiles. Assisted-by: Claude Code Co-Authored-By: Claude Opus 5.5 <[email protected]>WIP: fix(agent): a teleport the game never carries out is sent again, MinGit comes before the self-test, and broken lifted C is built againto fix(agent): a teleport the game never carries out is sent again, MinGit comes before the self-test, and broken lifted C is built againPossibly related, for context:
Recommendation: Request changes. The teleport retry biases the farm’s load-time measurements and can resend without confirming that the game is still on the source map. Windows can still claim a job before testing MinGit, and the build retry deletes cache entries used by other builds.
The three changes aim to save failed farm jobs by retrying stalled teleports, testing Windows Git before the first job, and rebuilding after generated C fails to compile. The recovery paths add waits to healthy runs, leave a Windows first-job failure path open, and discard more build cache than the failed build owns.
@fixwill do (18)agent/Core/ScenarioRunner.cs:1075: The retry sleeps a full second before its first status sample, and the caller only starts LoadTimeAsync after the retry returns. (more on the line)agent/Core/ScenarioRunner.cs:1083: The grace period can end with no current evidence that the teleport stalled. (more on the line)agent/Core/ScenarioRunner.cs:1089: A null target passes the different-map check. (more on the line)agent/Core/ScenarioRunner.cs:1046: The source map and refusal budget span all sends. (more on the line)agent/Core/ScenarioRunner.cs:1060: The final accepted send skips verification. (more on the line)agent/Core/ScenarioRunner.cs:1096: The new status calls can fail a healthy teleport before it is sent or after it is accepted. (more on the line)agent/Core/ScenarioRunner.cs:1072: The 20-second check uses wall-clock time and can start a one-second delay plus a ten-second request just before its deadline. (more on the line)agent/Core/WindowsSelfTest.cs:55: When GitHub is temporarily unavailable, the self-test can save an AppContainer choice with Git skipped. (more on the line)agent/Core/WindowsSelfTest.cs:53: The standalone sandbox-check command can now install MinGit at the same time as the GUI, serve, or a job. (more on the line)agent/Core/WindowsSelfTest.cs:55: An access-denied error creating or replacing the MinGit cache escapes the intended best-effort fetch path. (more on the line)agent/Core/Workspace.cs:1306: The retry matches a shortened display message. (more on the line)agent/Core/Workspace.cs:1313: A compiler error in one base removes the entire shared lift store. (more on the line)agent/Core/Workspace.cs:1329: The reported baseline build time only includes the second attempt. A warm build can spend minutes compiling before failing, then report only the cold retry time. Record both attempts or include the failed attempt and cleanup in the baseline total while keeping the successful attempt's phases separately identifiable.agent/tests/agent_core/TimeoutNoteChecks.cs:20: The tests call the status predicate, so deleting the resend from StepWithRetry would leave them green. (more on the line)agent/tests/agent_core/SetupChecks.cs:68: The new checks only match strings. (more on the line)docs/test-plan.md:578: The plan reference only describes retrying refusals. A plan author who puts a timer, input, or capture after a teleport now gets an extra observation window and possibly another send. Document the final retry policy and when the next step begins.For the author (1) (a run, a measurement or a decision)
Checks: Still running:
CI / Coordinator tests (pull_request),CI / Agent build and checks (pull_request),CI / Coordinator image (pull_request),CI / Conventional Commit title (pull_request),CI / Workflow lint (actionlint) (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven. The PR reports 61 teleport timeouts over three days on Windows and Linux, five build failures on one PC, and 2,310 passing checks in a container. The first two counts describe the existing failures. There are no matched base and head runs showing that the same plans recover, that load measurements stay accurate, or that the affected Windows sandbox chooses a working mode before the first job.
Other systems: Teleport handling runs on both sides of every Windows and Linux comparison, including Steam Deck. It changes load probes, plan timers, and when subsequent input or capture steps run. Windows setup, the standalone sandbox check, and startup refresh now share MinGit installation with jobs. The base retry affects the lift store read by PR packs and warm build benchmarks. The diff leaves the plan and report schemas, sandbox grants, save handling, and game code untouched.
Switches: None. The retry limits are internal constants. The existing LONGSHOT_LIFT_STORE and LONGSHOT_OBJ_STORE values are unchanged, but the retry clears the directory named by LONGSHOT_LIFT_STORE.
Reviewed:
c9cecfb2dfagainst345885fd1d: Read AGENTS.md, CONTRIBUTING.md, the full diff and commit message, WindowsSelfTest.cs, SetupChecks.cs, and TimeoutNoteChecks.cs. Read the changed methods in ScenarioRunner.cs and Workspace.cs in full, plus their relevant callers, bridge and process handling, build diagnostics, cache readers, sandbox selection, MinGit installation, and test registration. The rest of ScenarioRunner.cs and Workspace.cs was reviewed selectively. Skimmed the relevant plan and report docs and CI workflow. No review checklist, PR template, recorded run attachments, translations, or generated files were present in the change.; nothing run (a code review).Checked and fine:
Testing: Read-only review. No builds, tests, or project scripts were run. The PR reports 2,310 container checks passed. All five listed Forgejo checks were still running, with no reported failures and no tester runs in the supplied discussion.
Overlaps and merge order: None found in the checkout or supplied discussion.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -1044,0 +1043,4 @@// (a teleport the game takes but never carries out: it stays in play where it was, and the next step's wait for the map ran// out of time, 61 game starts in 3 days on main and PRs alike, Nov 2026. Sent again, up to TeleportSends in all)string? before = step.Do == "teleport" ? await MapNowAsync(bridge, cancel) : null;for (int attempt = 1, sends = 1; ; attempt++)L4: The source map and refusal budget span all sends. If the game moves to the target while an initial refusal is being retried, the saved source is stale when the next send is accepted. If two accepted sends stall, a refusal on the final send also has fewer retries than the original refusal path. Refresh the source for the send being verified and keep refusal retries separate from the cap on accepted teleports.
@ -1054,1 +1057,4 @@continue;}var target = (string?)r["map"] ?? step.To;if (step.Do != "teleport" || before == null || sends >= TeleportSends) return (at, (string?)r["map"]);L5: The final accepted send skips verification. If all three sends leave the game on the source map, a plan that goes straight from teleport to measure or shot can report from the wrong map. Observe the final send too and surface a teleport-specific failure if the game is still in play on the source map after the grace period.
@ -1058,0 +1069,4 @@/// Whether a teleport the game took has begun within TeleportGrace: the map is another, or the game is loadingstatic async Task<bool> TeleportStartedAsync(ControlClient bridge, RunningProcess game, string before, string? target, CancellationToken cancel){var until = DateTime.UtcNow + TeleportGrace;L7 (minor): The 20-second check uses wall-clock time and can start a one-second delay plus a ten-second request just before its deadline. A clock adjustment extends it further. Use elapsed time and cap both the delay and request by the remaining grace period.
@ -1058,0 +1072,4 @@var until = DateTime.UtcNow + TeleportGrace;while (DateTime.UtcNow < until){if (await Task.WhenAny(Task.Delay(TimeSpan.FromSeconds(1), cancel), game.Exited) == game.Exited) throw new GameDied();L1: The retry sleeps a full second before its first status sample, and the caller only starts LoadTimeAsync after the retry returns. A teleport that finishes in 300 ms can be reported as roughly one second. A transition that finishes before that first sample can also vanish from the probe when the reply omits its map. Use one observation loop for retry and load timing, sample immediately, and carry the first observed completion time through to the report.
@ -1058,0 +1080,4 @@}catch (Exception e) when (e is BridgeError) { }}return false;L2: The grace period can end with no current evidence that the teleport stalled. For example, if early polls show the old map and the late polls return bridge errors, the method still authorizes another send even if loading began in the meantime. Conversely, a reply with a missing map or an unrelated state counts as progress. Retry only on a fresh, valid sample showing play on the source map. Treat loading or a known map change as progress and treat unavailable or malformed status separately.
@ -1058,0 +1086,4 @@/// A teleport not carried out yet: still in play on the map it left (a teleport to the map it is on, or one to a map the/// game doesn't name, is never sent again)public static bool TeleportStalled(string before, string? mapNow, string? stateNow, string? target) =>!string.Equals(before, target, StringComparison.OrdinalIgnoreCase) && stateNow == "play"L3: A null target passes the different-map check. A plan using a destination index has no step.To, so an acknowledgement without a map can make the agent resend a teleport to the current map. Resolve the target before enabling retries or skip retries when the target cannot be determined.
@ -1058,0 +1093,4 @@{try{var st = await bridge.RequestAsync("status", null, TimeSpan.FromSeconds(10), cancel);L6: The new status calls can fail a healthy teleport before it is sent or after it is accepted. A status request that times out throws BridgeGone, which the new code does not handle and the caller can report as a game hang. The new casts also throw on a non-string map or state. Use the existing game-exit-aware request path, validate fields by type, and distinguish an optional status sample timing out from a terminal bridge failure.
@ -47,0 +50,4 @@try{using var fetcher = new HttpFetcher(TimeSpan.FromSeconds(60), TimeSpan.FromSeconds(120));await MinGit.EnsureAsync(Path.Combine(home.Data, "mingit"), Path.Combine(home.Data, ".cache", "mingit"), fetcher, say, cancel);L9: The standalone sandbox-check command can now install MinGit at the same time as the GUI, serve, or a job. The self-test gate is process-local and the standalone command bypasses the instance lock. Concurrent installers share the same .part and .unpack paths, so one can fail or remove staging files the other is using. Serialize installation by data directory across processes and recheck the installed pin once the lock is held.
@ -47,0 +52,4 @@using var fetcher = new HttpFetcher(TimeSpan.FromSeconds(60), TimeSpan.FromSeconds(120));await MinGit.EnsureAsync(Path.Combine(home.Data, "mingit"), Path.Combine(home.Data, ".cache", "mingit"), fetcher, say, cancel);}catch (Exception e) when (e is InfraException or IOException or HttpRequestException or FetchException) { say($"MinGit could not be fetched yet: {e.Message}"); }L8: When GitHub is temporarily unavailable, the self-test can save an AppContainer choice with Git skipped. That choice stays current while MinGit is missing. If the download then succeeds inside the first job, the job is already using the unchecked AppContainer and can still fail at NUL. Keep the sandbox choice provisional until the Git probe runs, and finish provisioning and refresh the choice before accepting a Windows job.
L10 (minor): An access-denied error creating or replacing the MinGit cache escapes the intended best-effort fetch path. A Windows PC with an unwritable cache loses the rest of its sandbox diagnostics. Handle UnauthorizedAccessException as a provisioning failure and let the other probes run while keeping the Git result provisional. Continue to propagate cancellation.
@ -1296,1 +1303,4 @@clock = await SandboxedAsync(variant.IsDefault ? "the main base" : $"the main base, {variant.Label}", "build-base", command, Longshot, home,readOnly, writable, progress, cancel, With(VariantEnv(variant), StoreEnv(LiftStore)));}catch (JudgeException e) when (warm && GeneratedCodeBroken(e.Message))L11: The retry matches a shortened display message. StageClock selects the last @@​error summary or the last 15 lines, and SandboxedAsync then keeps only 1,500 characters. A generated-C diagnostic followed by a longer traceback or a generic build summary disappears before the catch filter sees it. Record the matching compiler diagnostic as process output arrives and carry that result separately from the display text.
@ -1297,0 +1310,4 @@// nothing: a work folder the disk or the memory damaged is not kept)log($"main's generated C did not compile in the kept work folder: built again without it ({Last(e.Message, 200)})");DeleteTree(Path.Combine(baseDir, "work"));DeleteTree(LiftStore);L12: A compiler error in one base removes the entire shared lift store. For example, a failure building a variant can erase valid translations for a kept default base. That default base can later be reused without rebuilding it, leaving its next warm PR pack or benchmark to lift again. Retry in an isolated workspace with an empty lift store, or invalidate only the entry proven bad. The PR description currently explains deletion of the kept work, but omits eviction of the shared store.
@ -65,6 +65,11 @@ static partial class Program"other bytes than the pin's hash are refused");Require(e.Passing && !Directory.Exists(Path.Combine(tarDir, "other")), "refused before unpacking; passing (a bad download)");Require(Workspace.GeneratedCodeBroken(@"the main base failed (exit 1): build.py stopped (exit status 1): Y:\bases\main\work\sa\gen\code_004.c:112352:29: error: expected ']'"),L15: The new checks only match strings. Removing the rebuild entirely would still pass, as would removing MinGit provisioning from the self-test. Drive the build retry through a fake sandbox that emits a diagnostic followed by a generic summary, and test the Windows bootstrap with an injected fetcher that can succeed or defer. Assert the build calls, cache scope, and Git probe order without using the network.
@ -17,2 +17,4 @@ScenarioRunner.WhereWords(new JsonObject { ["state"] = "play", ["map"] = "KokiriForest_Child" }), "its state and map");Equal("the game was in loading, on no known map", ScenarioRunner.WhereWords(new JsonObject { ["state"] = "loading" }), "no map");// (a teleport the game took but never carried out is sent again: still in play on the map it left)Require(ScenarioRunner.TeleportStalled("KokiriForest_Child", "KokiriForest_Child", "play", "HyruleField_Child"), "still in play where it was: stalled");L14: The tests call the status predicate, so deleting the resend from StepWithRetry would leave them green. Exercise the actual request sequence with a fake bridge and controllable time. Check the send cap, final failure, same or unknown destination, transient status failures, and fast load timing.