Summary

When messages.visibleReplies is set to message_tool, the agent is expected to call the message tool to deliver replies. If the LLM forgets (a probabilistic event — the reporter saw it happen on the 8th turn after 7 correct deliveries), the reply was silently discarded: the substantive text sat in the JSONL transcript but was never delivered to the user.

The existing shouldWarnAboutPrivateMessageToolFinal() detection was already identifying these stranded replies and logging a warning, but took no recovery action.

Fix

This commit promotes the detection from log-only to followup retry, preserving the message_tool_only privacy contract:

1. Gate room events: Skip stranded reply detection for ambient room_event turns where message_tool_only is automatically set by the system. Silence on room events is intentional by design — the agent chose not to respond.

2. Enqueue followup retry turn: When a stranded reply is detected, enqueue a followup run whose prompt instructs the agent to deliver the captured text via message(action=send). The agent itself calls the message tool, so delivery happens through the correct contract path — no suppression bypass, no privacy boundary crossing.

3. Clear transcript/persistence fields: The retry run explicitly clears transcriptPrompt, userTurnTranscriptRecorder, and currentInboundContext from the original followup run to prevent the original user prompt from hiding the retry instruction. The followup runner resolves transcriptPrompt ?? extracted.text, so a retained transcriptPrompt would cause the agent to see the original user question instead of the retry delivery instruction.

4. No suppression bypass: This does NOT mark payloads with deliverDespiteSourceReplySuppression or use skipSuppressionCheck. The message_tool_only privacy contract is fully preserved: delivery only happens when the agent explicitly calls message(action=send).

Normal (non-stranded) paths are unaffected: short replies, intentional NO_REPLY, and runs where the message tool already delivered are all excluded by the existing shouldWarnAboutPrivateMessageToolFinal guard.

Why followup retry instead of bypass delivery?

The message_tool_only contract says: "no message tool call = private/silent". Auto-publishing stranded text would break this documented behavior. Instead, we give the agent a second chance to follow the contract correctly — like returning an undelivered letter and asking the sender to mail it properly, rather than opening and publishing it ourselves.

Changed files

• src/auto-reply/reply/agent-runner.ts — followup retry with cleaned transcript fields, room_event gate (+26/-3 net)
• src/auto-reply/reply/agent-runner.misc.runreplyagent.test.ts — 5 new retry/gate tests replacing 6 bypass tests (-4 net)

Tests

New test cases:

• enqueues a followup retry for a stranded substantive reply (#85714) — verifies enqueueFollowupRun is called with retry prompt containing the assistant text
• does not enqueue retry for short private final replies — negative case
• does not enqueue retry when the message tool delivered this turn — negative case
• does not enqueue retry for room_event turns (#85714) — verifies room_event gate
• clears transcriptPrompt on retry to prevent original prompt from hiding retry instruction (#85714) — regression test verifying transcriptPrompt, userTurnTranscriptRecorder, and currentInboundContext are all undefined on the retry run

Real behavior proof

• Behavior or issue addressed: Agent's final message silently discarded when LLM forgets to call the message tool in message_tool_only mode. The fix enqueues a followup retry turn so the agent can deliver via message(action=send), preserving the privacy contract.
• Real environment tested: Local macOS arm64, Node.js v24.10.0, patched checkout at 9dc3c76dc3.
• Privacy contract preserved: The fix does NOT use deliverDespiteSourceReplySuppression or skipSuppressionCheck. No payload is auto-delivered. The agent is re-prompted to call message(action=send) itself.
• Transcript isolation: The retry run clears transcriptPrompt, userTurnTranscriptRecorder, and currentInboundContext so the followup runner's transcriptPrompt ?? extracted.text resolution correctly uses the retry prompt, not the original user message.
• Room event safety: Stranded reply detection is gated by sessionCtx.InboundEventKind !== "room_event", so ambient room events (where silence is intentional) never trigger recovery. Covered by dedicated test case.
• Failing proof before fix: On upstream/main, warnPrivateMessageToolFinal only logs — no retry is enqueued and the reply is silently lost.
• Evidence after fix:

  ✓ auto-reply agent-runner.misc.runreplyagent.test.ts (47 tests) 1729ms
  Test Files  1 passed (1)
       Tests  47 passed (47)
  
  ✓ unit-fast private-message-tool-final.test.ts (8 tests) 2ms
  Test Files  1 passed (1)
       Tests  8 passed (8)
  
  ✓ auto-reply dispatch-from-config.test.ts (189 tests) 3770ms
  Test Files  1 passed (1)
       Tests  189 passed (189)
  
  ✓ auto-reply followup-runner.test.ts (67 tests) 1710ms
  Test Files  1 passed (1)
       Tests  67 passed (67)
  

• Observed result after fix: When a stranded reply is detected, enqueueFollowupRun is called with a retry prompt containing the original assistant text and instructions to call message(action=send). The retry run has transcriptPrompt: undefined so the followup runner correctly uses the retry instruction. Short replies, already-delivered replies, room events, and runs with existing transcript prompts are all verified by dedicated test cases.
• What was not tested: Live end-to-end delivery through an actual channel — the test verifies that the correct followup run is enqueued with the expected prompt shape and cleaned fields. The followup queue machinery (enqueueFollowupRun → scheduleFollowupDrain → runReplyAgent) is a well-tested existing mechanism used for concurrent message queuing.

Real behavior proof (end-to-end, qa-lab host runner)

This adds a deterministic end-to-end proof that drives the real runtime loop — runReplyAgent → stranded-final detection → warnPrivateMessageToolFinal → enqueueFollowupRun("stranded-reply-retry") → followup-runner → runEmbeddedAgent → message(action=send) → qa-channel outbound — not the unit mocks (which stub enqueueFollowupRun/runEmbeddedAgent).

• Behavior or issue addressed: Under messages.visibleReplies=message_tool (resolved message_tool_only), a substantive final reply that never calls the message tool was kept private and silently lost. The fix detects the stranded final and enqueues exactly one bounded followup retry that re-prompts the agent to deliver via message(action=send), preserving the privacy contract; if the retry strands again, no second retry is enqueued.
• Real environment tested: macOS Darwin 24.3.0 arm64, Node.js v24.10.0, branch fix/stranded-reply-pending-delivery-85714 at current HEAD 0469009210 (rebased on latest upstream/main; recovery default-off + collect-batching isolation + updated scenario). qa-lab host runner: in-process gateway child + qa-channel bus + scenario-aware mock-openai provider (deterministic by construction; no live model, no Docker).
• Exact steps or command run after this patch: ran the new scenario, PASS:

  node scripts/run-node.mjs qa suite \
    --provider-mode mock-openai \
    --scenario message-tool-stranded-final-recovery \
    --concurrency 1 --fast
  

• Evidence after fix: captured scenario result and the gateway's own runtime log line (ids redacted):

  COUNTS: { total: 1, passed: 1, failed: 0 }
  step: pass - recovered=1; stranded turns=1; retry delivery turns=1; WARN logged=true
  
  2026-06-06T06:31:23.411+08:00 [source-reply/private-final] agent produced a long private final reply without calling the configured delivery tool (message_tool_only); response kept private and not delivered to the source channel
  

  Asserted from the live request/outbound capture:
  • Turn 1 request: plannedToolName !== "message" (long plain-text final, stranded).
  • Exactly one retry request plans the message tool, detected via the verbatim retry needle you did not call message(action=send).
  • Retry delivery request: plannedToolName === "message", plannedToolArgs.action === "send", plannedToolArgs.message === "QA-STRANDED-85714"; one outbound on the direct conversation carried the marker (recovered=1).
  • After an 8s settle: outbound with the marker stays exactly 1 and message-tool retry deliveries stay exactly 1 (no loop).
• Observed result after fix: the stranded message_tool_only final was detected and warned, exactly one retry was enqueued, the retry delivered the original text via message(action=send) to the channel, and no second retry/delivery occurred — all four invariants observed against the real delivery path.
• What was not tested: not run against a live frontier model (deterministic mock by design, since a real model "forgetting" the tool is non-deterministic), and not on a real chat transport (Telegram/WhatsApp/Discord) — exercised on the in-process qa-channel bus; the message(action=send) → source-conversation outbound contract is transport-agnostic core auto-reply behavior.

Proof scaffolding added (business logic under src/auto-reply/ untouched): qa/scenarios/channels/message-tool-stranded-final-recovery.md (new scenario) and extensions/qa-lab/src/providers/mock-openai/server.ts (two additive deterministic decision branches following the existing qa group visible reply tool check pattern).

Closes #85714

Update — retry drain scheduling (follow-up)

Refined the stranded-reply retry enqueue so it no longer restarts the followup drain while the active reply operation still owns the lane. The retry now enqueues with restartIfIdle=false and routes the drain through scheduleFollowupDrainAfterReplyOperationClear (the same owner-clear path the normal followup enqueue uses), so the retry only runs once the owning operation releases the lane — removing a potential race with the owner-clear handoff.

• Changed: src/auto-reply/reply/agent-runner.ts — stranded retry enqueue mirrors the normal enqueue-followup path (dormant enqueue + deferred drain).
• Test: src/auto-reply/reply/agent-runner.misc.runreplyagent.test.ts — added schedules the stranded-reply retry drain only after the active reply operation clears, asserting the drain stays dormant while the reply operation owns the lane and fires only after it clears.
• Verification after this change (local, macOS arm64, Node v24.10.0):

  vitest run src/auto-reply/reply/agent-runner.misc.runreplyagent.test.ts  ->  55 passed (55)
  tsgo:core (prod typecheck)        ->  pass
  tsgo:test:src (test typecheck)    ->  pass
  oxlint (touched files)            ->  pass