Skip to content

fix(agents): recover genuine terminal child result on lost-context sweep (#90299)#92791

Closed
leorivastech wants to merge 1 commit into
openclaw:mainfrom
leorivastech:fix/issue-90299-terminal-result-predicate
Closed

fix(agents): recover genuine terminal child result on lost-context sweep (#90299)#92791
leorivastech wants to merge 1 commit into
openclaw:mainfrom
leorivastech:fix/issue-90299-terminal-result-predicate

Conversation

@leorivastech

@leorivastech leorivastech commented Jun 13, 2026

Copy link
Copy Markdown

Summary

Problem. Agent Teams could deliver a subagent completion event whose status was
failed: subagent run lost active execution context while the same event carried a
non-empty child result (the report shows a full # ARCHITECTURE.md). The lifecycle
status and the delivered result disagree, so a parent orchestrating with
sessions_spawn / sessions_yield treats successful work as a plain failure and can
retry, stall, or leave the workflow inconsistent.

Root cause. When the registry sweeper finds a stale active run whose in-memory
context is gone and whose session store has not settled a terminal status,
resolveCompletionFromSessionEntry() returns null and the sweep falls through to a
hardcoded lost-context error (subagent-registry.ts, the sweeper-lost-context
branch). It never consults the child transcript, even though the announce path has
already captured a real final result from it.

Fix. Before emitting the lost-context error, the sweeper now reconciles the child
transcript and, only when the child produced a genuine terminal result, settles the
run as ok so the lifecycle status matches the delivered result. When there is no
genuine result it keeps the existing lost-context error.

The success predicate is the key. Rather than reusing the broad display helper
readSubagentOutput (which also surfaces a synthetic "N tool call(s) made without visible output." summary) or a fresh "any visible text" check (which ignores a
trailing sessions_yield wait), this adds a strict predicate that reuses the existing
sessions_yield-aware snapshot
(summarizeSubagentOutputHistory) and returns a result
only when there is final visible assistant text and the child is not parked waiting.
This is exactly the boundary the two prior attempts (#90492, #91400) got wrong. The
predicate also rejects a toolUse assistant turn that carries visible pre-tool text
plus a tool call (the model expected to continue after tool results), and any trailing
tool-only turn after a text turn — in both cases the child is still working, not finished.

Changes

  • src/agents/subagent-announce-output.ts
    • Extract loadSubagentHistoryMessages() (shared history source).
    • Add resolveTerminalChildResultText(messages) — strict terminal-result predicate.
      Returns the final visible assistant text only; undefined for sessions_yield
      waits, toolUse turns (visible pre-tool text plus a tool call), trailing tool-only
      turns, tool-call-only histories, silent markers, and empty histories. A new
      lifecycle-only latestAssistantTurnTerminal snapshot flag tracks this; the display
      selector (selectSubagentOutputText / readSubagentOutput) ignores it, so display
      behavior is unchanged.
    • Add readTerminalChildResult(sessionKey, options) — reads a child session and
      applies the strict predicate (reuses the same source as readSubagentOutput).
  • src/agents/subagent-registry.ts
    • In the sweeper-lost-context branch, attempt readTerminalChildResult first; on a
      genuine result complete as ok (sweeper-lost-context-recovered). A read failure
      is caught and falls through to the existing lost-context error so the run is still
      settled (the sweep is never aborted).

What did NOT change (scope)

  • readSubagentOutput keeps its existing display semantics — only its internals were
    factored into a shared loader.
  • No announce-delivery, spawn, orphan-recovery, or Gateway API changes. Runs without a
    genuine terminal result still complete with the existing lost-context error.

Reproduction

  1. Run a task that uses sessions_spawn to create a subagent and sessions_yield to
    wait for completion.
  2. Arrange for the child to produce a final result while its in-process run context is
    cleared before the session store settles a terminal status (the sweeper then enters
    the lost-context branch).
  3. Before: the parent receives status: "error" /
    "subagent run lost active execution context" even though the same event carries the
    child's result.
  4. After: when a genuine terminal result is present, the parent receives
    status: "ok" with the result; tool-call-only and yield-wait children still settle
    as lost-context errors.

Real behavior proof

Behavior or issue addressed: The lost-context sweeper branch labeling a stale child
run as failed: subagent run lost active execution context while a real child result
was delivered, and the inverse risk of recovering a non-terminal child (tool-call-only
or sessions_yield wait) as a successful completion (#90299).

Real environment tested: Linux (Ubuntu 24.04, Node v22.22). A standalone harness
writes three real child transcripts as on-disk .jsonl files in OpenClaw's transcript
format, then drives the shipping reconciliation and output code against them: the real
resolveCompletionFromSessionEntry, the real transcript reader
(readSessionMessagesAsync), and the real output predicates. The child's in-memory run
context is genuinely absent and the session entry status genuinely lags as running, so
resolveCompletionFromSessionEntry returns null and the lost-context branch is
exercised exactly as in the report. Each predicate decision is computed by the real
shipping code over the real parsed transcript; the prior approaches (#90492's
readSubagentOutput and #91400's hasVisibleAssistantOutput) are run side by side over
the same files for comparison.

Exact steps or command run after this patch:

  • Write four real transcripts to a temp dir: child-terminal.jsonl (final assistant
    result), child-tool-only.jsonl (tool call, no visible result), child-yield.jsonl
    (visible progress text followed by a sessions_yield wait), and child-pretool.jsonl
    (visible pre-tool text plus a tool call, stopReason: "toolUse").
  • Run the harness: node_modules/.bin/tsx oc-proof-90299.ts.

Evidence after fix:

Trigger condition — session store reconciliation:
  resolveCompletionFromSessionEntry(status="running" lagging entry) => null
  => null  ⇒ the sweep falls through to the lost-context branch (as in the bug report).

SCENARIO: genuine terminal result (child delivered the deliverable)   correct: ok
  ORIGINAL  (main)                   : status=error ✗ WRONG  delivered child result IS present → status disagrees with result (the bug)
  PR #90492 (readSubagentOutput)     : status=ok    ✔
  PR #91400 (hasVisibleAssistant.)   : status=ok    ✔
  THIS FIX  (readTerminalChildResult): status=ok    ✔  recovered ("# ARCHITECTURE.md\n## System Overview\nPure Python"…)

SCENARIO: tool-call-only run (no visible result)                      correct: error
  ORIGINAL  (main)                   : status=error ✔
  PR #90492 (readSubagentOutput)     : status=ok    ✗ WRONG  false-ok via readSubagentOutput → "1 tool call(s) made without visible outp"…
  PR #91400 (hasVisibleAssistant.)   : status=error ✔
  THIS FIX  (readTerminalChildResult): status=error ✔  no genuine terminal result → kept lost-context

SCENARIO: visible text THEN sessions_yield wait (parked on descendants)  correct: error
  ORIGINAL  (main)                   : status=error ✔
  PR #90492 (readSubagentOutput)     : status=error ✔
  PR #91400 (hasVisibleAssistant.)   : status=ok    ✗ WRONG  recovered via hasVisibleAssistantOutput (ignores trailing yield)
  THIS FIX  (readTerminalChildResult): status=error ✔  no genuine terminal result → kept lost-context

SCENARIO: visible pre-tool text + tool call (toolUse, still working)  correct: error
  ORIGINAL  (main)                   : status=error ✔
  PR #90492 (readSubagentOutput)     : status=ok    ✗ WRONG  false-ok via readSubagentOutput → "Let me read RELEASE_BRIEF.md first."…
  PR #91400 (hasVisibleAssistant.)   : status=ok    ✗ WRONG  recovered via hasVisibleAssistantOutput
  THIS FIX  (readTerminalChildResult): status=error ✔  no genuine terminal result → kept lost-context

Observed result after fix: On the genuine-terminal-result transcript the run settles
as ok with the captured result instead of the contradictory failed: lost active execution context, resolving the reported disagreement. On the tool-call-only,
text-then-sessions_yield, and pre-tool-toolUse transcripts the run correctly stays on
the lost-context error path. Across the four real transcripts this is the only predicate
correct on all of them: the original main reports failed even with a delivered result;
readSubagentOutput falsely recovers the tool-call-only and pre-tool-toolUse runs;
hasVisibleAssistantOutput falsely recovers the yield-wait and pre-tool-toolUse runs.

What was not tested: A full live Gateway socket round-trip with a real LLM-backed
subagent process that loses context mid-run was not driven. The harness exercises the real
reconciliation, transcript parsing, and predicate code over real on-disk transcripts and a
real lagging session entry, but it synthesizes the lost-context window rather than killing
a live gateway process, and it does not depend on any external model/provider.

Notes

AI-assisted change. Automated coverage added and passing: unit tests for
resolveTerminalChildResultText / readTerminalChildResult (terminal result, tool-only,
sessions_yield wait, text-after-yield, toolUse pre-tool text, trailing tool-only,
whitespace, empty) and three sweeper integration tests in subagent-registry.test.ts
(recovers ok with a genuine result; keeps the lost-context error for tool-call-only and
for a toolUse pre-tool-text turn). Core and test typechecks are clean.

Fixes #90299. Supersedes the inactive #90492 (and the closed #91400), addressing the
predicate-breadth findings ClawSweeper raised on both: tool-call-only histories,
sessions_yield waits, and toolUse turns with visible pre-tool text are not recovered
as success, and a transcript-read failure preserves the lost-context error instead of
aborting the sweep.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: M proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 13, 2026
@clawsweeper

clawsweeper Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 13, 2026, 7:58 PM ET / 23:58 UTC.

Summary
The PR adds a strict terminal child-result reader for subagent transcripts and uses it in the lost-context sweeper branch, with unit and registry tests for terminal text, tool-only, sessions_yield, and toolUse cases.

PR surface: Source +100, Tests +320. Total +420 across 4 files.

Reproducibility: yes. Source inspection shows current main can fall from unresolved session-store reconciliation into the lost-context error while the announce path can still read child output; the PR body also includes a transcript-harness reproduction over real on-disk transcripts.

Review metrics: 1 noteworthy metric.

  • Lifecycle Recovery Path: 1 error-to-ok stale-run branch added. The PR changes terminal state and parent delivery semantics for stale subagent runs, so maintainers should consciously accept the recovery boundary before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] A false-positive terminal-result predicate would convert a genuinely incomplete or still-working child run from a lost-context error to ok, mis-stating parent/child session state.
  • [P1] The supplied proof exercises real transcript/session readers but synthesizes the context-loss timing window rather than killing a live Gateway-backed subagent process mid-run.

Maintainer options:

  1. Proceed With Strict Predicate (recommended)
    Accept the bounded lifecycle risk after exact-head checks because the PR rejects the known non-terminal transcript shapes and covers them in focused tests.
  2. Require Live Gateway Race Proof
    Ask for maintainer-run live Agent Teams proof if the team wants confidence beyond the transcript harness for the process-context-loss timing window.
  3. Keep the Older Error-Only Behavior
    Pause or close the PR only if maintainers decide stale lost-context recovery must never infer success from child transcript text.

Next step before merge

  • [P2] No repair lane is needed; the remaining action is normal maintainer merge judgment after exact-head checks.

Security
Cleared: The diff only changes agent lifecycle/output code and tests; it does not touch dependencies, workflows, secrets, permissions, or package execution paths.

Review details

Best possible solution:

Land the focused recovery after exact-head checks and maintainer acceptance, preserving the strict terminal predicate and unchanged display helper semantics.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main can fall from unresolved session-store reconciliation into the lost-context error while the announce path can still read child output; the PR body also includes a transcript-harness reproduction over real on-disk transcripts.

Is this the best way to solve the issue?

Yes. This is the narrowest maintainable fix I found: keep session-store terminal status authoritative, then recover only strict terminal transcript text before falling back to the existing lost-context error; the broader alternatives in #90492 and #91400 did not cover the same predicate boundaries.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against c2754150c963.

Label changes

Label justifications:

  • P2: This is a normal-priority Agent Teams lifecycle bugfix with limited blast radius but real session-state and delivery consequences.
  • merge-risk: 🚨 session-state: The PR can change a stale child run from failed to successful based on transcript state, so an incorrect predicate would leave lifecycle state inconsistent.
  • merge-risk: 🚨 message-delivery: The PR controls whether child transcript text is delivered as a completed result rather than as a failed lost-context event.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes after-fix terminal output from a Linux harness using real on-disk transcripts plus the shipping reconciliation and transcript reader code, which is sufficient for this non-visual lifecycle fix.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal output from a Linux harness using real on-disk transcripts plus the shipping reconciliation and transcript reader code, which is sufficient for this non-visual lifecycle fix.
Evidence reviewed

PR surface:

Source +100, Tests +320. Total +420 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 105 5 +100
Tests 2 320 0 +320
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 425 5 +420

What I checked:

  • Repository policy read: Read the full root AGENTS.md and the scoped src/agents/AGENTS.md; the deep-review, session-state risk, and agent-test guidance were relevant to this review. (AGENTS.md:1, c2754150c963)
  • Current main still has the lost-context error fallback: On current main, unresolved session-store reconciliation falls through to a hard error outcome: subagent run lost active execution context. (src/agents/subagent-registry.ts:937, c2754150c963)
  • Current main can attach output to a failed terminal outcome: The announce flow reads child output for failed terminal outcomes and then formats the event status from the error outcome, matching the reported failed-status plus result shape. (src/agents/subagent-announce.ts:398, c2754150c963)
  • Session-store reconciliation returns null for non-terminal running entries: resolveCompletionFromSessionEntry only returns completion for fresh terminal statuses; running or unsettled entries return null, which is the trigger for the sweeper fallback. (src/agents/subagent-session-reconciliation.ts:99, c2754150c963)
  • PR head adds a strict lifecycle-only terminal predicate: The PR-head predicate tracks latestAssistantTurnTerminal, rejects waiting/toolUse/tool-only shapes, and preserves display helper behavior separately from lifecycle recovery. (src/agents/subagent-announce-output.ts:145, 633a04aa16cc)
  • PR head recovers before the existing fallback and preserves fallback on no result/read failure: The sweeper reads readTerminalChildResult in the lost-context branch, completes ok only when a terminal result exists, and otherwise falls through to the existing lost-context error. (src/agents/subagent-registry.ts:962, 633a04aa16cc)

Likely related people:

  • steipete: Recent GitHub path history shows multiple commits in the subagent registry, lifecycle, output, and reconciliation surfaces that define this terminal-state behavior. (role: recent area contributor; confidence: high; commits: f24a13879095, 1e54e908e2e4, 5d81c29cc4f4; files: src/agents/subagent-registry.ts, src/agents/subagent-registry-lifecycle.ts, src/agents/subagent-announce-output.ts)
  • SebTardif: Commit 4980c32 recently touched failed subagent lifecycle recovery, the same fallback family this PR narrows. (role: completion-recovery contributor; confidence: medium; commits: 4980c32846c3, 907bc0371c3a; files: src/agents/subagent-registry.ts)
  • MonkeyLeeT: Commit 8a60f39 changed subagent run timeout enforcement across registry, lifecycle, and session reconciliation, overlapping this terminal outcome boundary. (role: adjacent terminal-contract contributor; confidence: medium; commits: 8a60f39221db; files: src/agents/subagent-registry.ts, src/agents/subagent-registry-lifecycle.ts, src/agents/subagent-session-reconciliation.ts)
  • openperf: Recent PR history shows a fix for yielded subagent terminal events in the same registry file, which is adjacent to this PR's sessions_yield terminal predicate risk. (role: recent adjacent contributor; confidence: medium; commits: cd3eb438f07f; files: src/agents/subagent-registry.ts)
  • vincentkoc: Recent registry and lifecycle commits touched subagent registry read caching and skipped delivery state, adjacent to stale-run cleanup and delivery behavior. (role: recent adjacent contributor; confidence: low; commits: c4a0ca0b7a41, 7e0d275f7add; files: src/agents/subagent-registry.ts, src/agents/subagent-registry-lifecycle.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jun 13, 2026
The registry sweeper labeled a stale, context-lost subagent run as
"failed: subagent run lost active execution context" even when the child
had delivered a genuine final result, so a parent orchestrating with
sessions_spawn / sessions_yield saw successful work as a plain failure.

Before emitting the lost-context error, reconcile the child transcript with
a strict terminal-result predicate that reuses the existing sessions_yield
-aware snapshot: recover as ok only when there is final visible assistant
text and the child is not parked waiting. Tool-call-only histories and yield
waits keep the lost-context error, and a transcript-read failure is caught so
the sweep still settles the run.

Fixes openclaw#90299.
@leorivastech
leorivastech force-pushed the fix/issue-90299-terminal-result-predicate branch from 88dc6b6 to 633a04a Compare June 13, 2026 21:09
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 13, 2026
@leorivastech

Copy link
Copy Markdown
Author

Thanks for the review — good catch on the toolUse boundary. Addressed the P1:

resolveTerminalChildResultText now rejects a non-terminal latest assistant turn. A genuine terminal result must be a visible-text assistant turn with no tool calls and no toolUse stop reason; a toolUse turn carrying visible pre-tool text, and any trailing tool-only turn after a text turn, are treated as non-terminal (the model expected to continue after tool results). This is tracked via a new lifecycle-only latestAssistantTurnTerminal snapshot flag — readSubagentOutput / selectSubagentOutputText display semantics are unchanged (they ignore the flag).

Added regression coverage: unit tests for a mixed text+toolCall (toolUse) turn and for a terminal text turn followed by a trailing tool-only turn, a reader test asserting the display helper still surfaces the pre-tool text while the lifecycle predicate returns undefined, and a sweeper integration test that keeps the lost-context error for a toolUse pre-tool-text child. The real-behavior harness in the PR body now includes the pre-tool-toolUse transcript as a fourth scenario.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jun 13, 2026
@vincentkoc vincentkoc self-assigned this Jun 14, 2026
@vincentkoc

Copy link
Copy Markdown
Member

closing this because transcript text alone is not a safe success predicate for lost-context lifecycle recovery.

Fresh autoreview found two blocking cases:

  • no-tool assistant turns with stopReason: error or aborted can contain partial/synthetic text and would be incorrectly settled as ok
  • the sweeper reads display-projected chat.history instead of the current run's recorded execution transcript, so it can miss the real result or recover stale/unrelated text from the same session key

The canonical fix must reconcile the current run's private transcript/status boundary and only accept an explicitly successful terminal turn. This patch is too risky to narrow safely without redesigning that recovery boundary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Agent Teams subagent completion can report "lost active execution context" while still delivering child output

2 participants