fix(agents): retire stale aborted subagent runs past the staleness window in orphan recovery (fixes #90766)#92346
Conversation
…ndow in orphan recovery (fixes openclaw#90766)
|
Thanks for the context here. I swept through the related work, and this is now duplicate or superseded. Close as superseded: the central useful fix is covered by the open, clean, proof-positive canonical PR, while this branch is conflicting, mock-only, and still missing the stronger finalization/docs coverage. Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Canonical path: Close this branch and continue review or landing through #90817 as the canonical fix for #90766. So I’m closing this here and keeping the remaining discussion on #90817 and #90766. Review detailsBest possible solution: Close this branch and continue review or landing through #90817 as the canonical fix for #90766. Do we have a high-confidence way to reproduce the issue? Yes at source level. Current main still has the abortedLastRun restore exemption, no orphan-recovery stale age gate, and a persistence test that expects a three-hour aborted restored run to be recovered through agent.wait. Is this the best way to solve the issue? No, this is not the best landing path as submitted. The safer solution is the clean canonical sibling that awaits stale finalization, updates docs, preserves fresh recovery, and includes restart-path proof. Security review: Security review cleared: No security or supply-chain concern found; the diff changes internal agent recovery logic and a colocated test only. AGENTS.md: found and applied where relevant. What I checked:
Likely related people:
Codex review notes: model internal, reasoning high; reviewed against a841c2788202. |
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
@clawsweeper re-review |
…le-orphan-recovery
…ubagent runs (fixes openclaw#90766) When a stale unended subagent run is detected, increment 'failed' instead of 'skipped' so scheduleOrphanRecovery can retry finalization. Previously the stale branch only incremented 'skipped' and the fire-and-forget Promise.allSettled finalization would silently drop errors. ClawSweeper P2: 'Make stale finalization retry-visible'
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@zenglingbiao thanks for the PR. ClawSweeper is still waiting on real behavior proof before this can move forward. Useful proof can be a screenshot, short video, terminal output, copied live output, linked artifact, or redacted logs that show the changed behavior after the fix. Please redact private tokens, phone numbers, private endpoints, customer data, and anything else sensitive. Once proof is added to the PR body or a comment, ClawSweeper or a maintainer can re-check it. |
Summary
abortedLastRun: trueare treated as resumable by orphan recovery even when they are hours past the staleness window. This can resurrect contextless child work long after the original run should have been retired.recoverOrphanedSubagentSessionschecksabortedLastRunand the recovery gate (re-wedge window) but never checks run age; (2)resolveSubagentRunOrphanReasonused in restore reconciliation exemptsabortedLastRun: trueentries from the stale-unended pruning branch (sessionEntry.abortedLastRun !== trueguard on line 193), so stale aborted runs survive registry init and proceed to orphan recovery.isStaleUnendedSubagentRuncheck inrecoverOrphanedSubagentSessionsafter theabortedLastRungate, beforeevaluateSubagentRecoveryGate— stale runs are skipped and reported asstale-unended-run.abortedLastRun !== trueexemption fromresolveSubagentRunOrphanReasonso restore reconciliation prunes stale runs regardless ofabortedLastRun. Fresh aborted runs (under the 2-hour staleness window) are NOT stale and still proceed to orphan recovery as before.src/agents/subagent-orphan-recovery.ts— importisStaleUnendedSubagentRun, add age-out guard before recovery gatesrc/agents/subagent-registry-helpers.ts— removeabortedLastRunexemption from stale-unended restore pruningsrc/agents/subagent-registry.persistence.test.ts— replaces the "keeps stale aborted runs for restart recovery" test with a "retires stale aborted runs past the staleness window" testevaluateSubagentRecoveryGate/subagent-recovery-state.ts— recovery gate unchanged; it handles re-wedge bounding, not stalenessdocs/tools/subagents.md) — out of scope; docs describe behavior at a higher levelSTALE_UNENDED_SUBAGENT_RUN_MScutoff still proceed to orphan recovery unchangedReproduction
agent.waitis called to resume itstale-unended-runReal behavior proof
Behavior or issue addressed (90766): Orphan recovery and restore reconciliation now age out stale aborted subagent runs past the
STALE_UNENDED_SUBAGENT_RUN_MS(2-hour) staleness window, instead of resurrecting them as restart-recoverable.Real environment tested: Linux, Node 22 — test runner against subagent orphan recovery, registry persistence, and registry queries
Exact steps or command run after this patch:
node scripts/run-vitest.mjs src/agents/subagent-orphan-recovery.test.ts src/agents/subagent-registry.persistence.test.ts src/agents/subagent-registry-queries.test.tsEvidence after fix:
Observed result after fix: The orphan recovery suite (16 tests including stale-skip coverage) and the persistence/queries suites (37 tests including the new "retires stale aborted restored runs past the staleness window" regression) all complete successfully. The before test on
origin/mainpasses the old "keeps stale aborted runs for restart recovery" behavior (3-hour-oldabortedLastRunrun stays recoverable), confirming the gap existed; the after test confirms those stale aborted runs are now retired and pruned.What was not tested: Live gateway restart with a real subagent run that spans multi-hour downtime and gate-level run-age cutoff. Systemd service stop/start cycle with a stale orphan.
Repro confirmation: The existing test on
origin/main(subagent-registry.persistence.test.tsline ~727, "keeps stale unended restored runs with abortedLastRun for restart recovery") seeds a 3-hour-old restored run withabortedLastRun: trueand expectscallGatewayto runagent.wait— confirming main treats the stale run as recoverable. After the patch, the replacement test seeds a 3-hour-oldabortedLastRunrun and confirms it is pruned from the registry (nocallGatewaycall, no announce). All other tests unchanged.Risk / Mitigation
abortedLastRunruns.Mitigation:
isStaleUnendedSubagentRunuses the existingSTALE_UNENDED_SUBAGENT_RUN_MS(2-hour) cutoff withresolveStaleCutoffMsrespecting explicitrunTimeoutSeconds. Freshly restarted runs under the cutoff are NOT stale and proceed to orphan recovery unchanged.abortedLastRunexemption from restore reconciliation could prematurely prune runs that should stay visible.Mitigation: The
isStaleUnendedSubagentRunguard is unchanged — it only fires when the run is past the staleness window. Runs under the window are not stale and stay in the registry regardless ofabortedLastRun.Change Type (select all)
Scope (select all touched areas)
Regression Test Plan
node scripts/run-vitest.mjs src/agents/subagent-orphan-recovery.test.tsnode scripts/run-vitest.mjs src/agents/subagent-registry.persistence.test.tsnode scripts/run-vitest.mjs src/agents/subagent-registry-queries.test.tsReview Findings Addressed
N/A (initial submission)
Linked Issue/PR
Fixes #90766