Skip to content

Subagent completion event sometimes drops result payload and/or token stats (3 repros, intermittent) #75196

Description

@DavidLeVineASC

Summary

Subagent runs (runtime: "subagent", agentId: "worker") that complete normally — committing to git and pushing to origin — sometimes deliver a completion event with an empty/zero result payload instead of the structured final-status block the worker actually produced. The work itself is fine; only the announce surface fails.

The pattern reproduces consistently on long-running fan-out work (LSBH fixture authoring at ~14-22 minute durations producing large output, ~60k tokens). Two confirmed reproductions across two separate fan-out batches in the ASCval modernization workspace.

Symptom

The supervisor receives an [Internal task completion event] with:

  • status: completed successfully
  • Stats: runtime <14-22m> · tokens 0 (in 0 / out 0)
  • A truncated assistant fragment (one or two sentences from mid-output) instead of the worker's structured final report

Counter-evidence the run actually succeeded:

  • All checkpoint commits are visible on origin/<branch> (verified via git log origin/<branch>)
  • All artifact files are on disk and pass downstream verification
  • A clean LSBH (or equivalent end-to-end) sweep against the worker's output passes

Reproductions

Reproduction 1 — Batch 3 round-1 W1 (mid-March-equivalent timestamp; ASCval session)

  • Task: author R1 archetype Track and persist Claude's returned session ID for conversation continuity #8 combo-db-dc (LSBH oracle authoring + generator + harness verification)
  • Worker commit progression: 56c5b539 step 1+2 (fixture 3/3 LSBH green) — pushed before the failed announce
  • Stats reported on completion event: tokens 0/0, runtime ~5 min
  • Output payload: a 0-byte combo-db-dc.gen.py cleanup artifact + a fragment with no final-status block
  • Originally interpreted as a real worker death; respawned in Batch 4 as W1-retry. The retry recognized state was already past steps 1-2 and only authored the missing step-3 oracles, confirming the "death" had been a successful run with a lost announce.

Reproduction 2 — Batch 5 W1 (2026-04-30 ~14:00 UTC)

  • Task: author R1 archetype sendMessageWeb() only handles images, should support audio/video/documents #9 sh-auto-enroll (similar LSBH oracle authoring profile)
  • Worker commit progression on origin/wave-phase-v-feature-a-probe-rollback:
    • 1fe9afa7 step 1 (1/1 LSBH green)
    • b623031e step 2 (3/3 LSBH green)
    • 08c239e6 step 3 (6/6 LSBH green + regression sweep) — pushed before the failed announce
  • Stats reported on completion event: tokens 0 (in 0 / out 0), runtime 14m 37s
  • Output payload: "I'll update the oracle to match the established dc-25 SH pattern (no formally registered Match formula → ACP=N/A). Updating the gen.py:" (one fragment, no final-status block)
  • Verified clean: 9-archetype LSBH regression sweep passed all green; 753/0/1 dotnet tests; the worker's full 6-oracle suite (plan-config-readback, form-5500-participant-information, vesting-status, compliance-top-heavy, compliance-adp-acp, compliance-coverage-410b) was correct and pre-pushed.

Reproduction 3 — Batch 6 W1 (2026-04-30 ~16:00 UTC)

  • Task: author R1 archetype Fix path traversal vulnerability in media server #10 ltpt (LTPT/SECURE Act §603 fixture authoring; full plumbing including ParticipantHoursHistory seeding + ltpt-listing oracle authoring + 5-oracle handler set)
  • Worker commit progression on origin/wave-phase-v-feature-a-probe-rollback:
    • 6aade34d step 1 (1/1 LSBH green)
    • 17528af6 step 2 (4/4 LSBH green)
    • 3d2aac2e step 3 (5/5 LSBH green + 9-archetype regression sweep) — pushed before the announce
  • Stats reported on completion event: tokens 0 (in 0 / out 0), runtime ~15 min
  • Output payload: in this case the structured final-status block was actually delivered intact (full ARCHETYPE-10 STATUS: block, defect-surfacing, per-checkpoint SHAs). Only the token-accounting was zeroed out. So the announce-payload body and the stats body appear to fail independently — this is a third reproduction with a slightly different signature than reproductions 1 and 2 (where the body itself was also truncated).

Counter-example — Batch 7 W2 (2026-04-30 ~17:00 UTC)

Same supervisor session, similar workload profile, no failure:

  • Task: author R1 archetype WhatsApp voice notes fail without explicit "audio/ogg; codecs=opus" mimetype #7 cash-balance (LSBH oracle authoring across ~30-participant fixture, two new run.mjs handlers, end-to-end actuarial validation)
  • Runtime: 16m 0s (within the 14-22m envelope of the failing reproductions)
  • Output: ~49.8k tokens (substantial, comparable to failing reproductions)
  • Result: clean. Full structured final-status block, accurate token stats (tokens 49.9k (in 76 / out 49.8k) • prompt/cache 126.8k). No truncation.

This counter-example confirms the failure is intermittent rather than a deterministic threshold — runtime and output size correlate but don't determine.

Contrast: same batch, different worker, no failure

Reproduction 2 happened in a two-worker concurrent fan-out. The other worker (W2 on archetype #4 dc-roth-loans) had a slightly longer run (20m 51s) producing more output (61.3k tokens) and delivered its full structured final-status block correctly, including detailed worker-review fields, defect-surfacing notes, and per-checkpoint commit SHAs. So this isn't a deterministic correlate of long runtime or large output — but those characteristics are present in both failing reproductions.

Hypothesis

Some interaction between (a) child runtime duration in the 14-22 minute range, (b) cumulative output size or token budget consumption, and (c) the announce-event delivery path causes the result payload to be truncated/dropped. The success-status flag itself appears unaffected (status: completed successfully is correct in the failed cases).

Possible angles for an investigator:

  • Result-payload buffer truncation when the worker's final assistant message is large or when there are many tool-call/result alternations
  • Race between the worker's terminal announce-write and parent-side event-receive when the worker's process exits
  • Token-usage telemetry path being on the same code path as the result-payload serialization (since both come back as 0/empty in the failed cases)

Manual workaround

When a subagent completion event arrives with tokens=0 and a truncated payload, do not auto-respawn. Instead:

  1. git fetch origin on the work branch
  2. git log origin/<branch> — check whether the worker's expected commits landed
  3. Verify any output artifacts via the work's normal end-to-end check (in our case: LSBH harness sweep)
  4. If everything landed cleanly, treat the run as successful with a missing announce; pick up downstream work as if the structured report had arrived

This workaround has been operationally reliable across both reproductions but obviously isn't a fix.

Environment

  • OpenClaw runtime: agent=main | host=openclaw-gateway--0000017-797dcf74d7-5bh8t | os=Linux 6.6.130.1-3.azl3 (x64) | node=v24.14.0
  • Model used: anthropic-47/claude-opus-4-7 for both supervisor and worker subagents
  • Spawn config: runtime: "subagent", agentId: "worker", mode: "run"
  • Workspace: ASCval modernization (engineering project; large workspace with substantial AGENTS.md context)

Suggested ask of the maintainer

Confirm whether the symptom matches anything in the runtime's known-issue list. If it's net-new, instrumenting the announce path to log final-payload size + a serialization checkpoint at child-exit would likely localize the failure quickly given the consistent reproduction profile.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions