Skip to content

Commit ae63a48

Browse files
100yenadmin100yenadmin
authored
fix(auto-reply): recover stranded message-tool finals by default (#99536)
In messages.visibleReplies "message_tool" sessions, a successful agent turn that produced a substantive private final without calling message(action=send) previously left the user with silence and only an operator log. The gateway now enqueues one protected front-of-queue retry prompting the model to deliver the reply, and falls back to a sanitized visible diagnostic when the retry cannot be enqueued or also strands. Queue overflow protection is unified with the in-flight-aware drop policy (skip in-flight or protected items, reject when nothing is droppable), rejected overflow no longer refreshes the drain debounce, heartbeat turns are excluded from recovery, and recovery retries no longer share the client turn's queued-turn lifecycle. Fixes #85714 Thanks to Eva (@100yenadmin) for the contribution.
1 parent 26d200c commit ae63a48

22 files changed

Lines changed: 1757 additions & 95 deletions

‎docs/gateway/config-channels.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -824,9 +824,9 @@ See the full channel index: [Channels](/channels).
824824

825825
Group messages default to **require mention** (metadata mention or safe regex patterns). Applies to WhatsApp, Telegram, Discord, Google Chat, and iMessage group chats.
826826

827-
Visible replies are controlled separately. Normal group, channel, and internal WebChat direct requests default to automatic final delivery: final assistant text posts through the legacy visible reply path. Opt into `messages.visibleReplies: "message_tool"` or `messages.groupChat.visibleReplies: "message_tool"` when visible output should only post after the agent calls `message(action=send)`. If the model returns final text without calling the message tool in an opted-in tool-only mode, that final text stays private and the gateway verbose log records suppressed payload metadata.
827+
Visible replies are controlled separately. Normal group, channel, and internal WebChat direct requests default to automatic final delivery: final assistant text posts through the legacy visible reply path. Opt into `messages.visibleReplies: "message_tool"` or `messages.groupChat.visibleReplies: "message_tool"` when visible output should only post after the agent calls `message(action=send)`. If the model returns a substantive final answer without calling the message tool in an opted-in tool-only mode, that final text stays private, the gateway verbose log records suppressed payload metadata, and OpenClaw enqueues one recovery retry asking the model to deliver the same reply via `message(action=send)`.
828828

829-
Tool-only visible replies require a model/runtime that reliably calls tools, and are recommended for shared ambient rooms on latest-generation models such as GPT 5.5. Some weaker models can answer final text but fail to understand that source-visible output must be sent with `message(action=send)`. For those models, use `"automatic"` so the final assistant turn is the visible reply path. If the session log shows assistant text with `didSendViaMessagingTool: false`, the model produced private final text instead of calling the message tool. Switch to a stronger tool-calling model for that channel, inspect the gateway verbose log for the suppressed payload summary, or set `messages.groupChat.visibleReplies: "automatic"` to use visible final replies for every group/channel request.
829+
Tool-only visible replies require a model/runtime that reliably calls tools, and are recommended for shared ambient rooms on latest-generation models such as GPT 5.5. Some weaker models can answer final text but fail to understand that source-visible output must be sent with `message(action=send)`. OpenClaw recovers the common stranded-final case by default only when the final is substantive, the source turn was not a room event, send policy did not deny delivery, and no source reply was already sent. Recovery is bounded to one retry; it suppresses persistence for the synthetic retry prompt and keeps that retry out of collect batching so it cannot merge with unrelated queued prompts. If the retry also strands or cannot be enqueued, OpenClaw delivers only a sanitized diagnostic such as "I generated a reply but could not deliver it to this chat. Please try again." The original private final text is never marked for automatic source delivery. For models that repeatedly strand replies, use `"automatic"` so the final assistant turn is the visible reply path, switch to a stronger tool-calling model, inspect the gateway verbose log for the suppressed payload summary, or set `messages.groupChat.visibleReplies: "automatic"` to use visible final replies for every group/channel request.
830830

831831
If the message tool is unavailable under the active tool policy, OpenClaw falls back to automatic visible replies instead of silently suppressing the response. `openclaw doctor` warns about this mismatch.
832832

@@ -836,9 +836,9 @@ This rule applies to normal agent final text. Plugin-owned conversation bindings
836836

837837
Symptom: a group/channel @mention shows the typing indicator and the gateway log reports `dispatch complete (queuedFinal=false, replies=0)`, but no message lands in the room. DMs to the same agent reply normally.
838838

839-
Cause: the group/channel visible-reply mode resolves to `"message_tool"`, so OpenClaw runs the turn but suppresses the final assistant text unless the agent calls `message(action=send)`. There is no `NO_REPLY` contract in this mode; no message-tool call means no source reply. There is no error because suppression is the configured behavior. Normal group and channel turns default to `"automatic"`, so this symptom only appears when `messages.groupChat.visibleReplies` (or global `messages.visibleReplies`) is explicitly set to `"message_tool"`. Harness `defaultVisibleReplies` does not apply here — the group/channel resolver ignores it; it only affects direct/source chats (the Codex harness suppresses direct-chat finals that way).
839+
Cause: the group/channel visible-reply mode resolves to `"message_tool"`, so OpenClaw runs the turn but suppresses final assistant text unless the agent calls `message(action=send)`. There is no `NO_REPLY` contract in this mode; no message-tool call means the original final text is private. For substantive source turns OpenClaw now attempts one guarded recovery retry; short notes, explicit silence, room events, send-policy-denied turns, and already delivered turns are not retried. Normal group and channel turns default to `"automatic"`, so this symptom only appears when `messages.groupChat.visibleReplies` (or global `messages.visibleReplies`) is explicitly set to `"message_tool"`. Harness `defaultVisibleReplies` does not apply here — the group/channel resolver ignores it; it only affects direct/source chats (the Codex harness suppresses direct-chat finals that way).
840840

841-
Fix: either pick a stronger tool-calling model, remove the explicit `"message_tool"` override to fall back to the `"automatic"` default, or set `messages.groupChat.visibleReplies: "automatic"` to force visible replies for every group/channel request. The gateway hot-reloads `messages` config after the file is saved; only restart the gateway when file watching or config reload is disabled in the deployment.
841+
Fix: either pick a stronger tool-calling model, remove the explicit `"message_tool"` override to fall back to the `"automatic"` default, or set `messages.groupChat.visibleReplies: "automatic"` to force visible replies for every group/channel request. A substantive stranded final should no longer end as silent success; it should either recover through one `message(action=send)` retry or show the sanitized delivery-failure diagnostic. The gateway hot-reloads `messages` config after the file is saved; only restart the gateway when file watching or config reload is disabled in the deployment.
842842

843843
**Mention types:**
844844

‎extensions/qa-lab/src/providers/mock-openai/server.test.ts‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -368,6 +368,74 @@ describe("qa mock openai server", () => {
368368
expect(text.match(/[.!?]+(?:\s|$)/g)).toHaveLength(2);
369369
});
370370

371+
it("recovers the stranded-final fixture by calling the message tool on the retry prompt", async () => {
372+
const server = await startMockServer();
373+
374+
const initialBody = await expectResponsesJson<{
375+
output?: Array<{ content?: Array<{ text?: string }> }>;
376+
}>(server, {
377+
stream: false,
378+
model: "gpt-5.5",
379+
tools: [MESSAGE_TOOL],
380+
input: [
381+
makeUserInput(
382+
"qa stranded final recovery check. Include `QA-STRANDED-85714` in a thorough multi-sentence answer, but do not call any tool yet.",
383+
),
384+
],
385+
});
386+
387+
const initialText = initialBody.output?.[0]?.content?.[0]?.text ?? "";
388+
expect(initialText).toContain("QA-STRANDED-85714");
389+
expect(initialText.length).toBeGreaterThanOrEqual(120);
390+
expect(outputItems(initialBody).some((item) => item.type === "function_call")).toBe(false);
391+
392+
const retryBody = await expectResponsesJson(server, {
393+
stream: false,
394+
model: "gpt-5.5",
395+
tools: [MESSAGE_TOOL],
396+
input: [
397+
makeUserInput(
398+
[
399+
"qa stranded final recovery check.",
400+
"Your previous reply was not delivered to the conversation because you did not call message(action=send).",
401+
initialText,
402+
].join(" "),
403+
),
404+
],
405+
});
406+
407+
const toolCall = outputToolCall(retryBody, "message");
408+
expect(outputToolArgsFromItem(toolCall)).toEqual({
409+
action: "send",
410+
message: "QA-STRANDED-85714",
411+
});
412+
});
413+
414+
it("keeps the retry-failure stranded-final fixture as text without a message tool call", async () => {
415+
const server = await startMockServer();
416+
417+
const body = await expectResponsesJson<{
418+
output?: Array<{ content?: Array<{ text?: string }> }>;
419+
}>(server, {
420+
stream: false,
421+
model: "gpt-5.5",
422+
tools: [MESSAGE_TOOL],
423+
input: [
424+
makeUserInput(
425+
[
426+
"Your previous reply was not delivered to the conversation because you did not call message(action=send).",
427+
"Include `QA-STRANDED-RETRY-FAIL-RAW` in a thorough multi-sentence answer, but do not call any tool.",
428+
].join(" "),
429+
),
430+
],
431+
});
432+
433+
const text = body.output?.[0]?.content?.[0]?.text ?? "";
434+
expect(text).toContain("QA-STRANDED-RETRY-FAIL-RAW");
435+
expect(text.length).toBeGreaterThanOrEqual(120);
436+
expect(outputItems(body).some((item) => item.type === "function_call")).toBe(false);
437+
});
438+
371439
it("keeps final-only marker preview deltas separate from the final answer", async () => {
372440
const server = await startMockServer({ finalOnlyMarkerPauseMs: 1 });
373441
const response = await fetch(`${server.baseUrl}/v1/responses`, {

‎extensions/qa-lab/src/providers/mock-openai/server.ts‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -171,6 +171,11 @@ const QA_TOOL_PROGRESS_PROMPT_RE = /tool progress qa check/i;
171171
const QA_GROUP_VISIBLE_REPLY_TOOL_PROMPT_RE = /qa group visible reply tool check/i;
172172
const QA_GROUP_MESSAGE_UNAVAILABLE_FALLBACK_PROMPT_RE =
173173
/qa group message unavailable fallback check/i;
174+
const QA_STRANDED_FINAL_RECOVERY_PROMPT_RE = /qa stranded final recovery check/i;
175+
const QA_STRANDED_FINAL_RETRY_FAILURE_PROMPT_RE = /qa stranded final retry failure check/i;
176+
const QA_STRANDED_FINAL_RETRY_PROMPT_RE = /you did not call message\(action=send\)/i;
177+
const QA_STRANDED_FINAL_RETRY_FAILURE_MARKER =
178+
"QA-STRANDED-RETRY-FAIL-RAW";
174179
const QA_TELEGRAM_CURRENT_SESSION_STATUS_PROMPT_RE = /telegram current session_status qa check/i;
175180
const QA_TELEGRAM_STREAM_SINGLE_MARKER = "QA-TELEGRAM-STREAM-SINGLE-OK";
176181
const QA_TELEGRAM_LONG_FINAL_THREE_CHUNK_PROMPT_RE = /telegram long final three chunk qa check/i;
@@ -195,6 +200,28 @@ const QA_WHATSAPP_REPLY_TO_BOT_TRIGGER_MARKER_RE =
195200
const QA_WHATSAPP_BATCHED_FINAL_MARKER_RE = /\bWHATSAPP_QA_BATCHED_FINAL_([A-Z0-9]+)\b/u;
196201
const QA_SUBAGENT_DIRECT_FALLBACK_PROMPT_RE = /subagent direct fallback qa check/i;
197202
const QA_SUBAGENT_DIRECT_FALLBACK_WORKER_RE = /subagent direct fallback worker/i;
203+
204+
function buildStrandedFinalRecoveryText(): string {
205+
return [
206+
"QA-STRANDED-85714 confirms this is a substantive private final reply that initially skipped the message tool.",
207+
"The reply is intentionally long enough to exercise message_tool_only stranded-final recovery before the retry delivers it visibly.",
208+
].join(" ");
209+
}
210+
211+
function buildStrandedFinalRetryFailureText(): string {
212+
return [
213+
"QA-STRANDED-RETRY-FAIL-RAW confirms this retry also produced a substantive private final reply instead of calling the message tool.",
214+
"This text must remain private so the gateway can deliver only its sanitized failure diagnostic to the source chat.",
215+
].join(" ");
216+
}
217+
218+
function isStrandedFinalRetryFailureRequest(allInputText: string): boolean {
219+
return (
220+
QA_STRANDED_FINAL_RETRY_FAILURE_PROMPT_RE.test(allInputText) ||
221+
(QA_STRANDED_FINAL_RETRY_PROMPT_RE.test(allInputText) &&
222+
allInputText.includes(QA_STRANDED_FINAL_RETRY_FAILURE_MARKER))
223+
);
224+
}
198225
const QA_SUBAGENT_DIRECT_FALLBACK_MARKER = "QA-SUBAGENT-DIRECT-FALLBACK-OK";
199226
const QA_IMAGE_GENERATION_PROMPT_RE =
200227
/image generation check|capability flip image check|\/tool\s+image_generate/i;
@@ -1496,6 +1523,14 @@ function buildAssistantText(
14961523
"The response is long enough to exercise message_tool_only private-final detection while remaining private to the agent transcript.",
14971524
].join(" ");
14981525
}
1526+
if (isStrandedFinalRetryFailureRequest(allInputText)) {
1527+
return buildStrandedFinalRetryFailureText();
1528+
}
1529+
if (QA_STRANDED_FINAL_RECOVERY_PROMPT_RE.test(allInputText)) {
1530+
return QA_STRANDED_FINAL_RETRY_PROMPT_RE.test(allInputText)
1531+
? "QA-STRANDED-85714"
1532+
: buildStrandedFinalRecoveryText();
1533+
}
14991534
if (/tool continuity check/i.test(prompt) && toolOutput) {
15001535
return `Protocol note: model switch handoff confirmed on ${model || "the requested model"}. QA mission from QA_KICKOFF_TASK.md still applies: understand this OpenClaw repo from source + docs before acting.`;
15011536
}
@@ -2519,6 +2554,21 @@ async function buildResponsesPayload(
25192554
},
25202555
]);
25212556
}
2557+
if (isStrandedFinalRetryFailureRequest(allInputText)) {
2558+
return buildAssistantEvents(buildStrandedFinalRetryFailureText());
2559+
}
2560+
if (QA_STRANDED_FINAL_RECOVERY_PROMPT_RE.test(allInputText)) {
2561+
if (QA_STRANDED_FINAL_RETRY_PROMPT_RE.test(allInputText)) {
2562+
if (!toolOutput && hasDeclaredTool(body, "message")) {
2563+
return buildToolCallEventsWithArgs("message", {
2564+
action: "send",
2565+
message: "QA-STRANDED-85714",
2566+
});
2567+
}
2568+
return buildAssistantEvents("");
2569+
}
2570+
return buildAssistantEvents(buildStrandedFinalRecoveryText());
2571+
}
25222572
if (QA_GROUP_VISIBLE_REPLY_TOOL_PROMPT_RE.test(allInputText)) {
25232573
const marker = exactMarkerDirective ?? exactReplyDirective ?? "QA-GROUP-TOOL-OK";
25242574
if (!toolOutput && hasDeclaredTool(body, "message")) {

‎extensions/qa-lab/src/scenario-catalog.test.ts‎

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -647,18 +647,22 @@ describe("qa scenario catalog", () => {
647647
const strandedConfig = readQaScenarioExecutionConfig("message-tool-stranded-final-reply") as
648648
| { requiredChannelDriver?: string; requiredProviderMode?: string }
649649
| undefined;
650+
const retryFailureConfig = readQaScenarioExecutionConfig(
651+
"message-tool-stranded-final-retry-failure",
652+
) as { requiredProviderMode?: string } | undefined;
650653
const stranded = readQaScenarioById("message-tool-stranded-final-reply");
651-
const strandedFlow = JSON.stringify(stranded.execution.flow);
654+
const retryFailure = readQaScenarioById("message-tool-stranded-final-retry-failure");
652655
const heartbeat = readQaScenarioById("commitments-heartbeat-target-none");
653656
const heartbeatFlow = JSON.stringify(heartbeat.execution.flow);
654657

655658
expect(strandedConfig?.requiredProviderMode).toBe("mock-openai");
656-
expect(strandedConfig?.requiredChannelDriver).toBe("qa-channel");
657-
expect(strandedFlow).toContain("this seeded scenario is mock-openai only");
658-
expect(strandedFlow).toContain("state.getSnapshot().events.slice(eventStartIndex)");
659-
expect(strandedFlow).toContain("message.deleted !== true");
660-
expect(strandedFlow).toContain("config.expectedMarker");
661-
expect(strandedFlow).not.toContain("waitForNoOutbound");
659+
expect(retryFailureConfig?.requiredProviderMode).toBe("mock-openai");
660+
expect(JSON.stringify(stranded.execution.flow)).toContain(
661+
"this seeded scenario is mock-openai only",
662+
);
663+
expect(JSON.stringify(retryFailure.execution.flow)).toContain(
664+
"this seeded scenario is mock-openai only",
665+
);
662666
expect(heartbeatFlow).toContain("sessionKey");
663667
expect(heartbeatFlow).toContain("commitmentOutbound.length === 0");
664668
expect(heartbeatFlow).not.toContain("waitForNoOutbound");

0 commit comments

Comments
 (0)