fix(announce): durable in-process queue fallback for direct-pending handoffs#90144
fix(announce): durable in-process queue fallback for direct-pending handoffs#90144bradfreels wants to merge 1 commit into
Conversation
…andoffs
When subagent direct-announce dispatch returns a non-terminal status
("accepted"/"started"/"in_flight") and the requester parent is not
streaming (yielded, queued, compaction-prep, or tool-result truncated),
the previous behaviour returned {delivered:false, error:"completion
agent handoff is still pending"}, surfacing as the dominant
`delivery_failed` audit-finding signature.
Route those cases into the requester's in-process durable system-event
inbox instead, keyed by the announce idempotency key. The inbox is
idempotent on (text, contextKey, deliveryContext), bounded at 20 events
per session, and drained on the parent's next turn-start (the same path
already used by jobs, hooks, restart sentinels, ACP child spawns,
monitor events, and the group/channel completion path in task-registry).
The `SubagentDeliveryPath` union gains a new `"durable_queue""` literal
so consumers can route on it. The harness-task-runtime durability check
(`isDurableAgentHarnessCompletionDelivery`) is extended to recognise the
new path, preventing Codex native-subagent monitor retry from churning
duplicates against an already-committed inbox enqueue.
If `enqueueSystemEvent` itself throws, the original give-up payload
(structured `reason: "completion_handoff_pending"` + error string) is
returned as a safety net so existing retry/give-up bookkeeping remains
the floor.
Tests:
- 5 unit tests in subagent-announce-delivery.test.ts covering the new
branch, idempotency on duplicate fires, no-fallback-on-active-stream,
no-fallback-with-media (preserve existing path), and result-envelope
shape.
- 1 integration test in embedded-agent-runner/sessions-yield.orchestration
reproducing the full failure class end-to-end: parent yield ->
isEmbeddedAgentRunActive(false) -> announce with stubbed callGateway
returning `accepted` -> assert path:"durable_queue" -> drain returns
the trigger.
- 1 unit test in agent-harness-task-runtime.test.ts asserting
durable_queue is treated as durable and that delivered:false still
fails fast.
Targeted suite (5 files): 150/150 pass.
Full unit lane: 10071 pass; the 5 pre-existing failures on main
(`status.summary.runtime`, `subagents-tool` config-isolation issues)
reproduce with no patch applied and are not regressions from this
change.
|
Thanks for the context here. I did a careful shell check against current Current main already fixes the direct-pending subagent completion handoff through the merged narrower fix in #94349, so this branch’s durable-queue implementation is now superseded and should not remain a landing candidate. Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. So I’m closing this as already implemented rather than keeping a duplicate issue open. Review detailsBest possible solution: Keep the merged narrow pending-status fix on main and leave any stronger durable-queue delivery contract to a separate maintainer-sponsored design if needed. Do we have a high-confidence way to reproduce the issue? Yes at source level: Is this the best way to solve the issue? No for this branch as submitted. The narrower merged current-main fix solves the reported handoff loss without adding the broader durable-queue/SDK surface that previously carried session-state and compatibility risk. Security review: Security review cleared: The PR diff only changes internal TypeScript agent delivery logic and tests; it does not touch dependencies, workflows, credentials, packaging, or external code execution. AGENTS.md: found and applied where relevant. What I checked:
Likely related people:
Codex review notes: model internal, reasoning high; reviewed against d4833e27c775; fix evidence: commit 95c87e31e295, main fix timestamp 2026-06-18T23:38:11Z. |
|
ClawSweeper applied the proposed close for this PR.
|
Summary
When
deliverSubagentAnnouncementdirect-dispatch returns a non-terminal status (accepted/started/in_flight) and the requester parent is not actively streaming (yielded, queued, compaction-prep, or tool-result truncated), today's behaviour returns— which surfaces as the dominant signature in
openclaw tasks audit --jsondelivery_failedfindings: the requester parent never receives a child-completion event that already arrived in the gateway. From inside the requester, the symptom is a subagent that appearsactive (waiting on N children)withpendingDescendants > 0indefinitely after the envelope'sendedAtalready fired, because the completion event can't deliver back.This PR routes those cases into the requester's existing in-process durable system-event inbox instead, keyed by the announce idempotency key. The inbox is the same primitive used today by jobs, hooks, restart sentinels, ACP child spawns, monitor events, and the group/channel completion path in task-registry. It is idempotent on
(text, contextKey, deliveryContext), bounded at 20 events per session, and drained on the parent's next turn-start.What changed
SubagentDeliveryPathunion insubagent-announce-dispatch.tsextended with a new"durable_queue"literal (with doc comment).subagent-announce-delivery.ts: thedirectAnnounceStillPendinggive-up branch (whenexpectsCompletionMessage && expectedMediaUrls.length === 0 && !requiresMessageToolDelivery) now callsenqueueSystemEventwithcontextKey: "subagent-announce:${params.directIdempotencyKey}"anddeliveryContext: requesterSessionOrigin ?? completionExternalFallbackOrigin ?? directOrigin, returning{ delivered: true, path: "durable_queue" }. ExistingdirectIdempotencyKeyis reused — no new param threading.enqueueSystemEventitself throws, the original give-up payload ({ delivered: false, path: "direct", reason: "completion_handoff_pending", error: ... }) is returned so today's retry/give-up bookkeeping remains the floor.isDurableAgentHarnessCompletionDeliveryinplugin-sdk/agent-harness-task-runtime.tsnow treats"durable_queue"as durable, preventing the Codex native-subagent monitor from looping retry against an inbox that already idempotently holds the event.Tail-risk:
SubagentDeliveryPathconsumersThree real consumers; all safe:
subagent-registry-lifecycle.tsif (delivery.path === "none")gated on!delivereddurable_queuereturnsdelivered:truesubagent-announce.tsif (!delivery.delivered && delivery.path === "direct" && delivery.error)!deliveredplugin-sdk/agent-harness-task-runtime.ts(isDurableAgentHarnessCompletionDelivery)falsefor any non-steered/directpathTests
subagent-announce-delivery.test.ts— covers the new branch firing, idempotency on duplicate fires, no-fallback-on-active-stream (steered path wins), no-fallback-with-media (existing direct/media path preserved), and the result-envelope shape downstream consumers see.embedded-agent-runner/sessions-yield.orchestration.test.ts— reproduces the failure class end-to-end: yielded parent →runEmbeddedAgentreturnsstopReason:"end_turn"→isEmbeddedAgentRunActive(parentSessionId)returnsfalse→deliverSubagentAnnouncementwith stubbedcallGatewayreturningaccepted→ assertspath:"durable_queue", inbox holds the trigger,drainSystemEventsreturns it, post-drain inbox empty.agent-harness-task-runtime.test.ts— assertsdurable_queueis treated as durable and thatdelivered:falsestill fails fast.Targeted suite (5 files): 150/150 pass.
Full unit lane (
pnpm test:unit): 10071 passed across 1055 files. The 5 failures observed are insrc/commands/status.summary.runtime.test.ts(3) andsrc/agents/tools/subagents-tool.test.ts(2) — both of which fail identically on a clean unpatched main HEAD (verified via stash + re-run). Cause appears to be a test-isolation issue where those tests load the host's~/.openclaw/openclaw.jsonrather than a sandboxed config; not related to this patch.pnpm tsgo:core,pnpm tsgo:test:src,pnpm exec oxfmt --check(6 files),pnpm exec oxlint(6 files): all clean.Not run
pnpm test:e2e,pnpm test:live,pnpm test:docker:*— require live provider credentials and/or docker daemon access. Thenot_streamingfailure class is exercised deterministically by the integration repro; patch surface is one branch in one direct-dispatch function plus one downstream consumer. Happy to run any of these on request from a maintainer who has the live secrets.Out of scope (deliberate)
delivery_failedaudit findings (so the Brad-side watchdog could attempt automated retry) is held for a follow-up PR after this core fix lands. Filed mentally; not in this diff.Origin / context
I wrote and operated a Brad-side audit watchdog (alert-only, 3-minute cadence) that has been catching every fresh
delivery_failedincident on this side over the past day. The watchdog will stay alert-only and be retired after this fix is deployed and proven stable across multiple subagent flows. This PR is the upstream-side root-cause fix corresponding to the watchdog's mitigation surface.