Skip to content

fix(subagents): require assistant ack for completion wake#92934

Closed
mushuiyu886 wants to merge 3 commits into
openclaw:mainfrom
mushuiyu886:feat/issue-92433
Closed

fix(subagents): require assistant ack for completion wake#92934
mushuiyu886 wants to merge 3 commits into
openclaw:mainfrom
mushuiyu886:feat/issue-92433

Conversation

@mushuiyu886

@mushuiyu886 mushuiyu886 commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: active requester completion handoffs treated a queued internal completion user message_end as delivered even when the requester ended before consuming it; the direct-failure fallback steering path also used transcript commit alone.
  • Solution: completion handoff wakes now opt into waiting for a following assistant response after transcript commit, including fallback steering after a completion direct-send failure.
  • What changed: added waitForAssistantResponseAfterTranscriptCommit to the embedded queue wait path and enabled it only for completion handoff steering.
  • What did NOT change: non-completion progress steering still resolves at the transcript commit boundary; channel transports, gateway send behavior, message-tool-only delivery, and timeout defaults were not changed.
  • Why it matters / User impact: completed subagent results should either be consumed by the active requester or remain eligible for existing fallback delivery, not silently disappear at requester shutdown.
  • Fixes [Bug]: Subagent completion silently dropped when announce steers into a requester run that ends before processing it #92433

Real behavior proof

  • Behavior or issue addressed: Subagent completion is no longer marked delivered just because the internal completion user message reached the requester transcript. The fixed boundaries are both queued completion user message_end -> requester agent_end -> no assistant response and completion direct send fails -> fallback steering; both now require requester assistant acknowledgement before reporting a steered completion as delivered.
  • Real environment tested: Local Linux worktree /media/vdc/code/ai/aispace/openclaw-worktrees/issue-92433, branch feat/issue-92433, head ef5a2f8fd7daedac01fd5e93eecab43111d57b81.
  • Exact steps or command run after this patch:
cd /media/vdc/code/ai/aispace/openclaw-worktrees/issue-92433 && OPENCLAW_REPO=/media/vdc/code/ai/aispace/openclaw-worktrees/issue-92433 OPENCLAW_EVIDENCE_DIR=/media/vdc/code/ai/aispace/openclaw-issue-92433-evidence node --import tsx /media/vdc/code/ai/aispace/openclaw-issue-92433-evidence/subagent-completion-active-run-queue-proof.mjs > /media/vdc/code/ai/aispace/openclaw-issue-92433-evidence/subagent-completion-active-run-queue-proof.out.json
cd /media/vdc/code/ai/aispace/openclaw-worktrees/issue-92433 && OPENCLAW_TEST_FAST=1 node scripts/run-vitest.mjs src/agents/subagent-announce-delivery.test.ts -t "requires requester assistant acknowledgement when completion direct failure fallback-steers"
cd /media/vdc/code/ai/aispace/openclaw-worktrees/issue-92433 && OPENCLAW_TEST_FAST=1 node scripts/run-vitest.mjs src/agents/subagent-announce-delivery.test.ts src/agents/subagent-announce-dispatch.test.ts src/agents/embedded-agent-runner/run/attempt.queue-message.test.ts
cd /media/vdc/code/ai/aispace/openclaw-worktrees/issue-92433 && node scripts/run-tsgo.mjs -p tsconfig.core.json --noEmit
cd /media/vdc/code/ai/aispace/openclaw-worktrees/issue-92433 && ./node_modules/.bin/oxfmt --check --threads=1 src/agents/subagent-announce-delivery.ts src/agents/subagent-announce-delivery.test.ts
git -C /media/vdc/code/ai/aispace/openclaw-worktrees/issue-92433 diff --check
  • Evidence after fix: The active-run proof runs outside Vitest and does not call testing.setDepsForTest(). It registers a requester in the production ACTIVE_EMBEDDED_RUNS registry, lets the default production deliverAgentHarnessTaskCompletion() path resolve requester activity, and lets queueEmbeddedAgentMessageWithOutcomeAsync() call the real active-run handle. The handle uses the production steerAndWaitForTranscriptCommit() wait logic, so queue/gateway outcomes are not injected.
{
  "proof": "subagent-completion-active-run-queue-ack",
  "head": "ef5a2f8fd7daedac01fd5e93eecab43111d57b81",
  "runtime": "node --import tsx; production deliverAgentHarnessTaskCompletion; production ACTIVE_EMBEDDED_RUNS registry; production queueEmbeddedAgentMessageWithOutcomeAsync; production steerAndWaitForTranscriptCommit; no testing.setDepsForTest and no injected queue/gateway outcome",
  "ackScenario": {
    "delivery": {
      "delivered": true,
      "path": "steered",
      "phases": [
        {
          "phase": "direct-primary",
          "delivered": true,
          "path": "steered"
        }
      ]
    }
  },
  "teardownScenario": {
    "queueOutcome": {
      "queued": false,
      "sessionId": "active-run-proof-teardown-session",
      "reason": "runtime_rejected",
      "gatewayHealth": "live",
      "errorMessage": "active session ended before queued steering message was consumed"
    }
  },
  "assertions": {
    "completionDeliveredAfterAssistantAck": true,
    "activeQueueUsedAssistantAckOption": true,
    "teardownBeforeAssistantDoesNotReportDelivered": true,
    "teardownFailureMentionsUnconsumed": true,
    "noInjectedQueueOrGatewayOutcome": true
  },
  "events": [
    {
      "event": "active-run-registered",
      "mode": "assistant-ack",
      "resolvedSessionId": "active-run-proof-ack-session",
      "isActive": true
    },
    {
      "event": "queue-handle-called",
      "mode": "assistant-ack",
      "waitForTranscriptCommit": true,
      "waitForAssistantResponseAfterTranscriptCommit": true
    },
    {
      "event": "queued-user-message-end",
      "mode": "assistant-ack"
    },
    {
      "event": "assistant-message-start",
      "mode": "assistant-ack"
    },
    {
      "event": "active-run-registered",
      "mode": "teardown-before-assistant",
      "resolvedSessionId": "active-run-proof-teardown-session",
      "isActive": true
    },
    {
      "event": "queue-handle-called",
      "mode": "teardown-before-assistant",
      "waitForTranscriptCommit": true,
      "waitForAssistantResponseAfterTranscriptCommit": true
    },
    {
      "event": "queued-user-message-end",
      "mode": "teardown-before-assistant"
    },
    {
      "event": "agent-end-before-assistant",
      "mode": "teardown-before-assistant"
    }
  ]
}
Test Files  2 passed (2)
Tests  2 passed | 200 skipped (202)
Test Files  3 passed (3)
Tests  118 passed (118)
[test] passed 1 Vitest shard in 18.79s
node scripts/run-tsgo.mjs -p tsconfig.core.json --noEmit exited 0 with no stdout/stderr output.
All matched files use the correct format.
Finished in 48ms on 2 files using 1 threads.
git diff --check exited 0 with no whitespace errors.
  • Observed result after fix: A completion handoff delivered through the production active-run queue returns delivered only after the queued completion is followed by requester assistant activity. The same production queue wait returns runtime_rejected with active session ended before queued steering message was consumed when the requester reaches agent_end after the queued completion but before any assistant response, so transcript durability alone is no longer treated as completion delivery. Ordinary non-completion steering still uses transcript-commit delivery.
  • What was not tested: No live Discord, Telegram, Slack, Desktop, Mantis, or other external channel transport proof was run in this environment. The active-run proof uses deterministic in-process requester transcript events against the production active-run registry and queue owner; it is not a full external Gateway/model-provider/channel run. It specifically removes the previous proof gap where queue/gateway delivery outcomes were injected.

Regression Test Plan

  • Target test file: src/agents/embedded-agent-runner/run/attempt.queue-message.test.ts, src/agents/subagent-announce-delivery.test.ts, src/agents/subagent-announce-dispatch.test.ts
  • Scenario locked in: queued completion user message_end -> requester agent_end -> no assistant response rejects the active completion wake as unconsumed, and completion direct-send failure fallback steering also waits for requester assistant acknowledgement.
  • Why this is the smallest reliable guardrail: The new acknowledgement option is opt-in and enabled only for completion handoff steering. It reuses existing queue outcome and fallback paths rather than adding channel-specific delivery logic.

Merge risk

  • Risk labels considered: clawsweeper:no-new-fix-pr, clawsweeper:needs-maintainer-review, clawsweeper:needs-product-decision, nonlocal channel/live proof boundary.
  • Risk explanation: The patch changes runtime completion acknowledgement semantics, so maintainer review should focus on whether assistant message_start/message_end is the intended completion-consumption boundary. The risk is contained because non-completion steering keeps its previous transcript-commit behavior and unsupported transcript-wait targets still fall back to best-effort wake options.
  • Why acceptable: The local proof exercises the production active-run queue owner and converts the false delivered signal into the already-supported fallback flow without changing public channel contracts.

Root Cause

  • Root cause: The active requester wake and fallback steering paths used waitForTranscriptCommit as delivery proof for subagent completion handoffs. That proves the internal completion message was committed as a queued user entry, but it does not prove the requester consumed the completion or produced a visible assistant response.
  • Why this is root-cause fix: The requester transcript event stream now distinguishes durability from consumption. A matching queued user message_end proves durability; a later assistant message_start/message_end proves requester consumption. If terminal requester state arrives first, the wake is unconfirmed and can fall through to existing fallback delivery.
  • Fix classification: Root-cause reliability fix for subagent completion delivery acknowledgement.
  • Maintainer-ready confidence: High for the active-run queue owner and source-level race covered here. The fix narrows the delivered boundary only for completion handoff paths, and related local runtime tests pass. External channel/live transport proof remains outside this local environment.
  • Patch quality notes: The diff is limited to subagent completion steering options and regression expectations. It does not add sleeps, widen channel transport behavior, or change the completion direct-first dispatch ordering.
  • Architecture / source-of-truth check: The active requester transcript stream is the source of truth. The patch changes the source-of-truth delivery boundary before downstream fallback decisions, rather than adding downstream string checks or transport-specific special cases.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 14, 2026
@clawsweeper

clawsweeper Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 14, 2026, 2:08 PM ET / 18:08 UTC.

Summary
The PR adds an opt-in active-run queue wait for requester assistant activity after transcript commit and enables it for subagent completion handoff steering and direct-failure fallback steering.

PR surface: Source +53, Tests +102. Total +155 across 5 files.

Reproducibility: yes. at source level: current main resolves the active queue wait at the matching queued user message_end, and dispatch maps that steered outcome to delivered. The linked stable report supplies the real-run interleaving where requester agent_end follows without an assistant response.

Review metrics: 1 noteworthy metric.

  • Completion acknowledgement paths: 2 paths changed. Both active completion wake and direct-failure fallback steering now decide whether completion delivery is consumed or recoverable.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Get maintainer acceptance that assistant message_start/message_end is the desired completion-consumption signal before merge.

Risk before merge

  • [P1] The PR changes completion delivery semantics: a queued completion that commits but gets no requester assistant response will now remain undelivered or fall back after the delivery wait instead of being marked delivered immediately.
  • [P1] Assistant message_start or message_end is a plausible consumption signal, but maintainers still need to accept it over a teardown flush or durable adoption marker for exactly-once completion handoff semantics.

Maintainer options:

  1. Accept Assistant-Ack Boundary
    If maintainers agree that a following assistant response proves requester consumption, merge with the supplied terminal proof and focused regression coverage.
  2. Revise To Teardown Recovery
    If assistant response is too indirect, ask for a revised patch that routes committed-but-unconsumed completion events through direct fallback when requester teardown wins the race.
  3. Pause For Lifecycle Design
    If completion adoption needs durable exactly-once state across transcript and subagent bookkeeping, pause this PR and settle that design on the linked issue first.

Next step before merge

  • [P2] Human review owns the remaining acknowledgement-boundary choice; I found no narrow mechanical code defect for ClawSweeper to repair.

Security
Cleared: Cleared: the diff changes internal agent queue acknowledgement code and focused tests only; no dependency, CI, secret, permission, package, or supply-chain surface changed.

Review details

Best possible solution:

Land the narrow assistant-ack handoff if maintainers accept assistant response events as the completion-consumption boundary; otherwise revise toward teardown recovery or a durable adoption marker.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: current main resolves the active queue wait at the matching queued user message_end, and dispatch maps that steered outcome to delivered. The linked stable report supplies the real-run interleaving where requester agent_end follows without an assistant response.

Is this the best way to solve the issue?

Yes, conditionally: the PR fixes the implicated owner path without changing non-completion steering, but the assistant-response adoption signal still needs maintainer acceptance. If that signal is rejected, teardown recovery or a durable adoption marker is the safer direction.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 81924cfd5e61.

Label changes

Label justifications:

  • P1: The linked bug can silently lose subagent completions in active multi-agent workflows.
  • merge-risk: 🚨 message-delivery: The diff changes when a subagent completion is considered delivered versus eligible for fallback.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body supplies terminal output from a local production-path active-run proof on the PR head, including assistant-ack success and no-assistant teardown rejection.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body supplies terminal output from a local production-path active-run proof on the PR head, including assistant-ack success and no-assistant teardown rejection.
Evidence reviewed

PR surface:

Source +53, Tests +102. Total +155 across 5 files.

View PR surface stats
Area Files Added Removed Net
Source 3 60 7 +53
Tests 2 102 0 +102
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 5 162 7 +155

What I checked:

Likely related people:

  • joshavant: Authored and merged the prior subagent completion announce timing work that established the transcript-commit acknowledgement path now being tightened. (role: recent feature owner; confidence: high; commits: 903d9c13f3ee, dcd984e2af05, 3e2ac05cdf3f; files: src/agents/subagent-announce-delivery.ts, src/agents/subagent-announce-dispatch.ts, src/agents/subagent-registry-lifecycle.ts)
  • steipete: Recent commits in the same subagent announce delivery area changed active-wake retry timeouts and subagent ownership routing, both adjacent to this completion handoff path. (role: recent adjacent delivery contributor; confidence: medium; commits: db549137d3bb, 56b8c7a6a012, 6823dd589899; files: src/agents/subagent-announce-delivery.ts, src/agents/subagent-announce-delivery.test.ts, src/agents/subagent-spawn.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. labels Jun 14, 2026
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 14, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jun 14, 2026
@mushuiyu886

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 14, 2026
@vincentkoc vincentkoc self-assigned this Jun 14, 2026

@vincentkoc vincentkoc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep review found two replay/acknowledgement blockers in src/agents/embedded-agent-runner/run/attempt.queue-message.ts:

  • Any assistant start/end event currently satisfies acknowledgement, including synthetic error/aborted messages, so failed delivery can be reported as consumed.
  • After the completion is committed, a timeout still returns runtime_rejected; the fallback then replays the same completion while the original run may still finish, duplicating responses or side effects.

Best fix: require a successful non-aborted acknowledgement, and model post-commit pending delivery as an irreversible distinct outcome or abort/drain the original run before any replay.

@vincentkoc

Copy link
Copy Markdown
Member

Closing because the reported completion-delivery bug is valid, but this acknowledgement design is not safe to land and needs a lifecycle-contract rewrite rather than incremental patching.

Three blockers remain:

  • A timeout after transcript commit is ambiguous: the original run remains active, but this path becomes retryable and can replay the same completion through handoff/fallback, duplicating replies or side effects.
  • Synthetic assistant error/aborted events can satisfy the proposed acknowledgement even though the completion was not successfully consumed.
  • Reply-run and unsupported-wait sibling paths bypass the acknowledgement contract and can still false-deliver.

The safe shape is a closed delivery outcome such as not_committed, adopted, terminal_unconsumed, and committed_pending. committed_pending must suppress replay and remain owned by pending-delivery reconciliation; only definitively unconsumed terminal outcomes may fallback exactly once.

Keeping #92433 open for a canonical lifecycle fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Subagent completion silently dropped when announce steers into a requester run that ends before processing it

2 participants