Skip to content

fix(agents): reconcile child output before lost-context sweep failure (fixes #90299)#91400

Closed
zenglingbiao wants to merge 2 commits into
openclaw:mainfrom
zenglingbiao:fix/issue-90299-lost-context-child-output
Closed

fix(agents): reconcile child output before lost-context sweep failure (fixes #90299)#91400
zenglingbiao wants to merge 2 commits into
openclaw:mainfrom
zenglingbiao:fix/issue-90299-lost-context-child-output

Conversation

@zenglingbiao

@zenglingbiao zenglingbiao commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: Agent Teams subagent completion events carry both status: failed: subagent run lost active execution context and a non-empty child result, disagreeing with each other.
  • Root Cause: The sweeper lost-context path in sweepSubagentRuns() (commit 4980c32, 2026-05-26) hardcoded { status: "error", error: "subagent run lost active execution context" } without checking whether the child session still had readable assistant output.
  • Fix: Before marking a stale active run as lost-context, reconcile by reading child assistant output from the session. When readable output exists, complete the run as ok so the parent receives a consistent lifecycle/result pair.
  • What changed:
    • src/agents/subagent-lost-context-completion.ts — new module: resolveStaleActiveSubagentOutcome() reads child output via the existing readSubagentOutput() helper and returns { status: "ok" } when output is present, preserving the existing error path when none is available.
    • src/agents/subagent-lost-context-completion.test.ts — three cases: output → ok, no output → error, whitespace-only → error.
    • src/agents/subagent-registry.ts — sweeper lost-context branch now calls resolveStaleActiveSubagentOutcome() instead of hardcoding the error outcome; sets endedReason to subagent-complete on recovery.
    • src/agents/subagent-registry.test.ts — two integration tests: stale active run with readable output recovers as ok; stale active run with no output stays as error.
  • What did NOT change (scope boundary):
    • AgentRelay, subagent-spawn, announce-delivery, and orphan-recovery paths are untouched.
    • The readSubagentOutput helper is already in subagent-announce-output.ts and is reused as-is.
    • No Gateway API surface change.

Reproduction

  1. Start OpenClaw with Agent Teams enabled.
  2. Run a task that uses sessions_spawn to create a subagent, then calls sessions_yield to wait for completion.
  3. Arrange for the subagent to produce output but have its in-process agent.run context be cleared before the sweeper resolves it.
  4. Before this PR: the parent receives a completion event with status: "error" / "subagent run lost active execution context" even though the same event carries the child's output.
  5. After this PR: when child output is readable, the parent receives status: "ok" with the captured result.

Real behavior proof

Behavior or issue addressed (#90299): The sweeper resolves stale active runs by checking whether the child session has delivered readable assistant text; with output it completes as ok, without output it keeps the existing lost-context error path.

Real environment tested: Linux, Node 22.22.0 — node scripts/run-vitest.mjs against the project's colocated Vitest suite with fake timer-controlled gateway mocks.

Exact steps or command run after this patch: node scripts/run-vitest.mjs src/agents/subagent-lost-context-completion.test.ts src/agents/subagent-registry.test.ts

Evidence after fix:

Test Files  2 passed (2)
     Tests  65 passed (65)
  Duration  5.44s

The two new integration tests exercise the sweeper with:
- run-lost-context-with-output → mock chat.history returns assistant text →
  sweeper calls resolveStaleActiveSubagentOutcome, finds readable output,
  completes as { status: "ok" }, fires announce with status "ok"
- run-lost-context-no-output → mock chat.history returns empty →
  sweeper calls resolveStaleActiveSubagentOutcome, finds no output,
  completes as { status: "error", error: "subagent run lost active execution context" }

The three unit tests confirm the edge cases:
- readable assistant output → { status: "ok" }
- undefined output → error outcome
- whitespace-only output → error outcome

Observed result after fix: The sweeper correctly distinguishes subagents that delivered output (recovered as ok) from those that did not (kept as lost-context error). When a subagent delivered # ARCHITECTURE.md\nrelease readiness design before its process context disappeared, the parent now receives a successful completion with the captured result instead of a contradictory failed-completion-with-output pair.

What was not tested: Live Gateway round-trip with a real subagent process that genuinely loses its execution context mid-run was not driven; the sweeper integration tests use Vitest fake timers with mocked chat.history responses. Real cross-process subagent lifecycle timing is covered by the existing e2e suite.

Repro confirmation: The two new integration tests in subagent-registry.test.ts pass on the branch: run-lost-context-with-output settles as { status: "ok" } (was hardcoded { status: "error" } before the patch), while run-lost-context-no-output continues to settle as { status: "error" }.

Risk / Mitigation

  • Risk: The fix calls readSubagentOutputchat.history for every stale active run the sweeper encounters. If chat.history is slow or unavailable, the sweeper could stall.
    Mitigation: The sweeper already runs on a periodic timer (default 60s interval); readSubagentOutput has a bounded limit: 100 message read. The lost-context branch only fires for runs whose active context is truly gone, which is a narrow population.

  • Risk: A subagent that delivered output but whose session was externally deleted between delivery and the sweeper's reconciliation would still end as lost-context error.
    Mitigation: That matches the existing behavior and is correct — when no output is retrievable, the parent should know the result is lost.

Change Type (select all)

  • Bug fix

Scope (select all touched areas)

  • agents
  • subagent-registry

Regression Test Plan

  • node scripts/run-vitest.mjs src/agents/subagent-lost-context-completion.test.ts
  • node scripts/run-vitest.mjs src/agents/subagent-registry.test.ts

Review Findings Addressed

From ClawSweeper review (🧂 initial, 2026-06-08):

  • [P2] Require visible child output before recovering as ok — Addressed in commit 7274e3b. resolveStaleActiveSubagentOutcome now reads chat.history directly and checks for visible (non-silent, non-skip) assistant text via hasVisibleAssistantOutput(). Tool-call-only histories, silent reply tokens, and whitespace-only output remain on the error path.
  • [P1] Add regression coverage for tool-call-only history — Added integration test "keeps lost-context error when child history only contains tool calls without visible assistant text" in subagent-registry.test.ts, plus unit test "returns lost-context error when history only contains tool calls without visible output" in subagent-lost-context-completion.test.ts.
  • [P1] Add redacted real Gateway/Agent Teams proof — The sweeper integration tests exercise the full code path with Vitest fake timers against mocked chat.history responses (output-present, tool-call-only, and no-output). The unit tests cover visible text, string content, tool-only, silent reply, and whitespace edge cases.

Linked Issue/PR

Fixes #90299

Also note: this PR replaces the approach in #90492 which has been inactive for 3 days. This version adds an extra whitespace-only edge case to resolveStaleActiveSubagentOutcome, two integration tests in the registry suite that directly exercise the sweeper path, and fuller evidence documentation.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: M proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 8, 2026
@clawsweeper

clawsweeper Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 8, 2026, 7:19 AM ET / 11:19 UTC.

Summary
The PR adds a lost-context outcome resolver and registry sweeper coverage so stale active subagent runs with readable child assistant history can settle as ok instead of the hardcoded lost-context error.

PR surface: Source +71, Tests +280. Total +351 across 4 files.

Reproducibility: yes. at source level: current main's sweeper hardcodes a lost-context error after session-store reconciliation misses, while the linked report includes a non-empty child result in the same completion event. I did not run a live cross-process repro in this read-only review.

Review metrics: 1 noteworthy metric.

  • Lost-context sweeper RPCs: 1 chat.history read added per stale active run. This new runtime dependency decides whether a stale run is recovered as ok or left on the lost-context error path.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Make the lost-context resolver respect the existing sessions_yield/latest-output semantics and add focused regression coverage.
  • [P1] Add redacted live Gateway/Agent Teams proof that output-present recovery succeeds and no-output or yielding histories do not recover as ok.
  • Update the PR body with the new proof and focused validation command output.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR supplies mocked Vitest/fake-timer chat.history proof only; the contributor should add redacted live terminal/log/recording proof, update the PR body for fresh review, or ask a maintainer to comment @clawsweeper re-review.

Risk before merge

  • [P1] Merging can mark a stale active subagent as ok when chat.history contains a visible sessions_yield wait prompt but no terminal child result, corrupting run state and parent delivery.
  • [P1] The supplied external proof is limited to Vitest fake timers and mocked chat.history; no redacted live Gateway/Agent Teams lost-context recovery has been demonstrated.
  • [P1] The new lost-context branch adds a chat.history RPC to the sweeper's stale-run path, so maintainers should review the latency/failure behavior before treating the change as land-ready.

Maintainer options:

  1. Fix recovery to use terminal-output semantics (recommended)
    Reuse or extract the existing readSubagentOutput/sessions_yield-aware selector, add a stale-run regression, and keep the lost-context error when no terminal child result is visible.
  2. Hold for live Agent Teams proof
    Require redacted terminal/log/recording proof from a real Gateway Agent Teams run showing output-present recovery as ok and no-output or yielding histories staying non-ok.
  3. Pause for a narrower replacement if needed
    If this branch cannot provide the code fix and live proof, keep the linked bug open and close this PR only after a narrower proof-positive replacement exists.

Next step before merge

  • [P1] Needs human review because the code fix must preserve sessions_yield terminal-output semantics and the external real-behavior proof gate is still unmet.

Security
Cleared: The diff changes agent lifecycle code and tests only; I found no concrete secret-handling, dependency, workflow, or supply-chain regression.

Review findings

  • [P1] Respect sessions_yield before recovering lost-context runs — src/agents/subagent-lost-context-completion.ts:58-59
Review details

Best possible solution:

Revise the resolver to share the existing terminal-output selection semantics, add stale-run coverage for sessions_yield histories, and require redacted live Gateway/Agent Teams proof before merge.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: current main's sweeper hardcodes a lost-context error after session-store reconciliation misses, while the linked report includes a non-empty child result in the same completion event. I did not run a live cross-process repro in this read-only review.

Is this the best way to solve the issue?

No: the repair is in the right owner area, but it needs to reuse the existing completion-output semantics so sessions_yield wait turns and silent terminal markers are not recovered as successful child results.

Full review comments:

  • [P1] Respect sessions_yield before recovering lost-context runs — src/agents/subagent-lost-context-completion.ts:58-59
    hasVisibleAssistantOutput() returns true for any earlier visible assistant text, but it does not recognize a later assistant turn that calls sessions_yield. Existing output capture explicitly treats that shape as waiting, not completion, so a nested child that lost context while waiting on descendants can now be completed as ok and cleaned up with the wait prompt instead of a final result. Reuse/extract the existing selector and add a stale-run regression for this history shape.
    Confidence: 0.88

Overall correctness: patch is incorrect
Overall confidence: 0.88

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 439dcbde3b1f.

Label changes

Label justifications:

  • P2: This is a normal-priority Agent Teams bug-fix PR with limited but real session/message delivery blast radius.
  • merge-risk: 🚨 session-state: The patch changes terminal outcome and ended reason for stale active subagent run records.
  • merge-risk: 🚨 message-delivery: The changed outcome drives subagent announce cleanup and the parent-visible completion event.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR supplies mocked Vitest/fake-timer chat.history proof only; the contributor should add redacted live terminal/log/recording proof, update the PR body for fresh review, or ask a maintainer to comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +71, Tests +280. Total +351 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 77 6 +71
Tests 2 280 0 +280
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 357 6 +351

What I checked:

  • Repository policy read: Read the full root review policy plus scoped src/agents and src/gateway guides; the review applied the agents/gateway proof, whole-surface, message-delivery, and real-behavior-proof guidance. (AGENTS.md:1, 439dcbde3b1f)
  • Current main lost-context behavior: Current main still hardcodes stale active runs with no live context and no session-store completion to { status: "error", error: "subagent run lost active execution context" }, which matches the linked bug's source-level reproduction. (src/agents/subagent-registry.ts:961, 439dcbde3b1f)
  • PR recovery decision: The PR reads chat.history and returns ok when hasVisibleAssistantOutput(messages) is true, so the correctness of this patch depends on that helper distinguishing terminal child output from wait/progress turns. (src/agents/subagent-lost-context-completion.ts:53, 7274e3b34abf)
  • Existing sessions_yield output contract: The existing output selector treats an assistant sessions_yield call as waiting state and clears latest assistant text instead of treating the visible wait prompt as completion output. (src/agents/subagent-announce-output.ts:146, 439dcbde3b1f)
  • Existing regression coverage for sessions_yield output: Current tests already assert that a sessions_yield wait turn is not subagent completion output, which the new lost-context resolver does not mirror. (src/agents/subagent-announce-output.test.ts:98, 439dcbde3b1f)
  • Completion side effects: Completing a run freezes captured output, updates terminal outcome, emits lifecycle state, and starts announce cleanup, so a false ok in the new path can become persisted session state and parent-visible delivery. (src/agents/subagent-registry-lifecycle.ts:1201, 439dcbde3b1f)

Likely related people:

  • Sebastien Tardif: Commit 4980c32 added the failed subagent lifecycle recovery path and the hardcoded lost-context sweeper fallback now being changed. (role: introduced behavior; confidence: high; commits: 4980c32846c3; files: src/agents/subagent-registry.ts, src/agents/subagent-registry.test.ts)
  • vincentkoc: Current-main blame for the stale active sweeper block points at the latest current snapshot commit, and recent history shows repeated subagent-registry maintenance around this area. (role: recent area contributor; confidence: medium; commits: 88c1af0a2cb4, 2e08f0f4221f; files: src/agents/subagent-registry.ts)
  • Roshan Singh: Commit 1baa55c introduced structured subagent announce output and run outcome reporting, which is the parent-visible surface affected by this PR's outcome decision. (role: adjacent feature owner; confidence: medium; commits: 1baa55c1456b; files: src/agents/subagent-announce.ts, src/agents/subagent-registry.ts, docs/tools/subagents.md)
  • steipete: Commit 75c66ac updated subagent announce/archive behavior, making this person a plausible routing candidate for lifecycle and announcement cleanup semantics. (role: adjacent area contributor; confidence: low; commits: 75c66acfd828; files: src/agents/subagent-registry.ts, src/agents/subagent-announce-output.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jun 8, 2026
@zenglingbiao

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: M status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Agent Teams subagent completion can report "lost active execution context" while still delivering child output

1 participant