Skip to content

fix(agent): suppress tool-use assistant text#71589

Closed
deepkilo wants to merge 7 commits into
openclaw:mainfrom
deepkilo:fix/feishu-group-final-only
Closed

fix(agent): suppress tool-use assistant text#71589
deepkilo wants to merge 7 commits into
openclaw:mainfrom
deepkilo:fix/feishu-group-final-only

Conversation

@deepkilo

Copy link
Copy Markdown
Contributor

Summary

  • suppress assistant messages that are tool-use continuations (stopReason: "toolUse" or tool-call content) from user-visible assistant output
  • keep explicit final_answer phase output deliverable
  • add regression coverage for unphased tool-use assistant text so it does not emit block/partial replies or final assistant text

Fixes #71588.

Notes

The Feishu group trigger policy (groupPolicy=open / requireMention=false) can make this easier to hit because more group messages start agent runs, but it is not the direct root cause. The direct issue is one agent run exposing both intermediate tool-use assistant text and the later final answer as user-visible replies.

Validation

  • corepack pnpm vitest run src/agents/pi-embedded-subscribe.subscribe-embedded-pi-session.suppresses-commentary-phase-output.test.ts
  • corepack pnpm tsgo:core
  • corepack pnpm exec oxlint --tsconfig tsconfig.oxlint.core.json src/agents/pi-embedded-subscribe.handlers.messages.ts src/agents/pi-embedded-subscribe.subscribe-embedded-pi-session.suppresses-commentary-phase-output.test.ts
  • corepack pnpm exec oxfmt --check src/agents/pi-embedded-subscribe.handlers.messages.ts src/agents/pi-embedded-subscribe.subscribe-embedded-pi-session.suppresses-commentary-phase-output.test.ts

Full corepack pnpm lint:core currently fails on an unrelated pre-existing no-array-sort finding in src/agents/cli-runner.spawn.test.ts:1029.

@chatgpt-codex-connector

Copy link
Copy Markdown
Contributor

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S labels Apr 25, 2026
@greptile-apps

greptile-apps Bot commented Apr 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a bug where agent runs were emitting both intermediate tool-use assistant text and the final answer as user-visible replies. It expands shouldSuppressAssistantVisibleOutput to handle unphased messages by checking for stopReason: "toolUse" or tool-call content blocks, and updates the mid-stream second gate in handleMessageUpdate to use the same predicate. A regression test is added for the unphased tool-use path.

Confidence Score: 4/5

Safe to merge; the fix is well-scoped and the two P2 findings are non-blocking.

Only P2-level findings: a gap in test coverage for the text_end channel and a theoretical miss of snake_case content-type variants. The core suppression logic is correct and the existing commentary phase behaviour is preserved.

No files require special attention beyond the P2 notes above.

Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-subscribe.subscribe-embedded-pi-session.suppresses-commentary-phase-output.test.ts
Line: 10-40

Comment:
**Missing `blockReplyBreak: "text_end"` coverage**

The new regression test only exercises the `blockReplyBreak: "message_end"` path. The `handleMessageUpdate` code has a separate `text_end`-channel flush at lines 617–634 (`ctx.blockChunker?.drain` and the deferred `flushBlockReplyBuffer`). A parallel fixture with `blockReplyBreak: "text_end"` would confirm those paths are also suppressed for unphased tool-use messages and guard against future regressions there.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/agents/pi-embedded-subscribe.handlers.messages.ts
Line: 40

Comment:
**Snake-case content-type variants not covered**

`TOOL_CALL_CONTENT_TYPES` contains `"toolCall"`, `"toolUse"`, and `"functionCall"` (camelCase). Some providers (e.g. OpenAI-compatible adapters) may emit `"tool_call"`, `"tool_use"`, or `"function_call"` (snake_case). If any upstream adapter normalises content block types to snake_case before emitting events, the content-block branch of `hasToolUseContinuation` would miss them. The `stopReason === "toolUse"` path would still fire, so this only matters when `stopReason` is absent and the detection falls back to content scanning.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "fix(agent): suppress tool-use assistant ..." | Re-trigger Greptile

Comment on lines +10 to +40
it("suppresses assistant messages that continue into tool use without phase metadata", () => {
const onBlockReply = vi.fn();
const onPartialReply = vi.fn();
const { emit, subscription } = createSubscribedSessionHarness({
runId: "run",
onBlockReply,
onPartialReply,
blockReplyBreak: "message_end",
});

const toolUseMessage = {
role: "assistant",
content: [
{ type: "text", text: "I'll check this now." },
{ type: "toolCall", id: "call-1", name: "read", input: { path: "README.md" } },
],
stopReason: "toolUse",
} as AssistantMessage;

emit({ type: "message_start", message: toolUseMessage });
emit({
type: "message_update",
message: toolUseMessage,
assistantMessageEvent: { type: "text_delta", delta: "I'll check this now." },
});
emit({ type: "message_end", message: toolUseMessage });

expect(onBlockReply).not.toHaveBeenCalled();
expect(onPartialReply).not.toHaveBeenCalled();
expect(subscription.assistantTexts).toEqual([]);
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing blockReplyBreak: "text_end" coverage

The new regression test only exercises the blockReplyBreak: "message_end" path. The handleMessageUpdate code has a separate text_end-channel flush at lines 617–634 (ctx.blockChunker?.drain and the deferred flushBlockReplyBuffer). A parallel fixture with blockReplyBreak: "text_end" would confirm those paths are also suppressed for unphased tool-use messages and guard against future regressions there.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-subscribe.subscribe-embedded-pi-session.suppresses-commentary-phase-output.test.ts
Line: 10-40

Comment:
**Missing `blockReplyBreak: "text_end"` coverage**

The new regression test only exercises the `blockReplyBreak: "message_end"` path. The `handleMessageUpdate` code has a separate `text_end`-channel flush at lines 617–634 (`ctx.blockChunker?.drain` and the deferred `flushBlockReplyBuffer`). A parallel fixture with `blockReplyBreak: "text_end"` would confirm those paths are also suppressed for unphased tool-use messages and guard against future regressions there.

How can I resolve this? If you propose a fix, please make it concise.

@@ -37,8 +37,37 @@ import {
promoteThinkingTagsToBlocks,
} from "./pi-embedded-utils.js";

const TOOL_CALL_CONTENT_TYPES = new Set(["toolCall", "toolUse", "functionCall"]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Snake-case content-type variants not covered

TOOL_CALL_CONTENT_TYPES contains "toolCall", "toolUse", and "functionCall" (camelCase). Some providers (e.g. OpenAI-compatible adapters) may emit "tool_call", "tool_use", or "function_call" (snake_case). If any upstream adapter normalises content block types to snake_case before emitting events, the content-block branch of hasToolUseContinuation would miss them. The stopReason === "toolUse" path would still fire, so this only matters when stopReason is absent and the detection falls back to content scanning.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-subscribe.handlers.messages.ts
Line: 40

Comment:
**Snake-case content-type variants not covered**

`TOOL_CALL_CONTENT_TYPES` contains `"toolCall"`, `"toolUse"`, and `"functionCall"` (camelCase). Some providers (e.g. OpenAI-compatible adapters) may emit `"tool_call"`, `"tool_use"`, or `"function_call"` (snake_case). If any upstream adapter normalises content block types to snake_case before emitting events, the content-block branch of `hasToolUseContinuation` would miss them. The `stopReason === "toolUse"` path would still fire, so this only matters when `stopReason` is absent and the detection falls back to content scanning.

How can I resolve this? If you propose a fix, please make it concise.

@deepkilo
deepkilo force-pushed the fix/feishu-group-final-only branch from 8aa46ee to 83f7a5d Compare April 25, 2026 13:48
@deepkilo

Copy link
Copy Markdown
Contributor Author

Rebased this branch on current main to pick up unrelated CI fixes that landed upstream after the previous run (notably the Array#toSorted() lint fix).\n\nNew head: 83f7a5dc1a.\n\nLocal validation after rebase:\n- corepack pnpm vitest run src/agents/pi-embedded-subscribe.subscribe-embedded-pi-session.suppresses-commentary-phase-output.test.ts\n- corepack pnpm tsgo:core\n- corepack pnpm lint:core\n- corepack pnpm exec oxfmt --check src/agents/pi-embedded-subscribe.handlers.messages.ts src/agents/pi-embedded-subscribe.subscribe-embedded-pi-session.suppresses-commentary-phase-output.test.ts

@deepkilo

Copy link
Copy Markdown
Contributor Author

Pushed a follow-up commit: e8f55349fe (fix(agent): clear suppressed commentary buffers).

This addresses the remaining gap behind #71588. The existing suppression path covers commentary/tool-use assistant text once the phase is already resolved, but it does not fully cover the delayed-phase stream case:

  • text_delta can arrive before the stream exposes phase: "commentary" via text_end / textSignature.
  • In that window, the visible assistant text may already have been appended into assistantTexts, blockBuffer, or partial/block chunking state.
  • Returning early in handleMessageEnd is then too late: the user-visible buffers can already be contaminated, which can still produce an intermediate/tool-use-looking reply alongside the final answer.

The new commit keeps the narrow suppression behavior, but additionally clears suppressed delta/block/chunker buffers and rolls assistantTexts back to the pre-block baseline when a commentary/tool-use assistant block is suppressed. It also expands the tool-call block guard to include snake_case variants (tool_call, tool_use, function_call).

Local validation:

  • corepack pnpm vitest run src/agents/pi-embedded-subscribe.subscribe-embedded-pi-session.suppresses-commentary-phase-output.test.ts
  • corepack pnpm tsgo:core
  • corepack pnpm lint:core
  • corepack pnpm exec oxfmt --check src/agents/pi-embedded-subscribe.handlers.messages.ts src/agents/pi-embedded-subscribe.subscribe-embedded-pi-session.suppresses-commentary-phase-output.test.ts

@deepkilo
deepkilo force-pushed the fix/feishu-group-final-only branch 2 times, most recently from ba51b30 to 144cc6c Compare April 26, 2026 06:06
@openclaw-barnacle openclaw-barnacle Bot added the app: macos App: macos label Apr 26, 2026
@deepkilo
deepkilo force-pushed the fix/feishu-group-final-only branch from 144cc6c to 3d3a202 Compare April 26, 2026 06:23
@openclaw-barnacle openclaw-barnacle Bot added size: M and removed app: macos App: macos size: S labels Apr 26, 2026
@clawsweeper

clawsweeper Bot commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Keep this PR open, but it is not merge-ready: the useful bug fix is still relevant on current main, yet the branch conflicts with renamed current code, lacks real channel proof, and still has concrete message-delivery gaps.

Canonical path: Close this stale PR. The latest review rated it F, the branch still lacks merge-ready proof, and there has been no human follow-up after the durable review.

So I’m closing this here because the remaining work is already tracked in the canonical issue.

Review details

Best possible solution:

Close this stale PR. The latest review rated it F, the branch still lacks merge-ready proof, and there has been no human follow-up after the durable review.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main only suppresses commentary-phase messages, and the PR head can still emit an unphased text_delta through partial reply delivery before later commentary/tool-use classification.

Is this the best way to solve the issue?

No. The handler is the right surface, but this branch is stale against current renamed code and still needs pre-delivery quarantine plus broader structured tool-call detection.

Security review:

Security review cleared: No concrete security or supply-chain regression was found; the lockfile change only adds a workspace importer and the remaining concern is functional message delivery.

AGENTS.md: found and applied where relevant.

What I checked:

  • stale F-rated PR: PR was opened 2026-04-25T13:10:19Z, is older than 30 days, and the latest review rated it F.
  • proof blocker: real behavior proof is missing and proof tier is F, so this branch is not merge-ready without contributor follow-up.
  • no human follow-up: live comments and timeline hydrated by apply contain no non-automation activity after the ClawSweeper review.

Likely related people:

  • mbelinky: Authored and merged the commentary-phase output suppression PR that changed the same subscribe handler and regression tests. (role: introduced adjacent suppression behavior; confidence: high; commits: 4175caee6e95; files: src/agents/pi-embedded-subscribe.handlers.messages.ts, src/agents/pi-embedded-subscribe.subscribe-embedded-pi-session.suppresses-commentary-phase-output.test.ts)
  • obviyus: Recent history on the same partial streaming filter surface includes fixes for partial streaming behavior and Telegram draft streaming partials. (role: partial-streaming area contributor; confidence: medium; commits: b5c2b1880d8c, 37721ebd7ca9; files: src/agents/embedded-agent-subscribe.ts, src/agents/embedded-agent-subscribe.handlers.messages.ts)
  • steipete: Split the embedded subscribe/tools code and appears repeatedly in history for reply payload and streaming delivery plumbing around this surface. (role: adjacent owner by refactor history; confidence: medium; commits: e2f89099829c, 62edfdffbdd0; files: src/agents/embedded-agent-subscribe.ts, src/agents/embedded-agent-subscribe.handlers.messages.ts)

Codex review notes: model internal, reasoning high; reviewed against 66880a5d736c.

@openclaw-barnacle openclaw-barnacle Bot added extensions: qa-lab channel: telegram Channel integration: telegram commands Command implementations labels Apr 26, 2026
@deepkilo
deepkilo force-pushed the fix/feishu-group-final-only branch from 844c674 to edddde9 Compare April 26, 2026 08:22
@openclaw-barnacle openclaw-barnacle Bot removed the channel: telegram Channel integration: telegram label Apr 26, 2026
@deepkilo
deepkilo force-pushed the fix/feishu-group-final-only branch from d02542f to 07b0b62 Compare April 26, 2026 08:32
@deepkilo
deepkilo force-pushed the fix/feishu-group-final-only branch from 07b0b62 to a6f2ffc Compare April 26, 2026 09:30
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. labels May 20, 2026
@clawsweeper

clawsweeper Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?
  • The egg game starts only after the PR passes the real-behavior proof check.
  • Before that, no creature or rarity is rolled. The treat waits for real proof.
  • This is still just collectible flavor: proof affects review readiness, not creature quality.

@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 20, 2026
@clawsweeper clawsweeper Bot added the mantis: telegram-visible-proof Mantis should capture Telegram visible proof. label May 20, 2026
@openclaw-barnacle

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jun 14, 2026
@clawsweeper

clawsweeper Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper applied the proposed close for this PR.

@clawsweeper clawsweeper Bot closed this Jun 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling commands Command implementations extensions: qa-lab mantis: telegram-visible-proof Mantis should capture Telegram visible proof. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: M stale Marked as stale due to inactivity status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feishu group can double reply when tool-use turns emit visible assistant text

1 participant