Skip to content

fix(outbound): sanitize assistant reasoning before delivery#84441

Closed
spinnakerbuilding-debug wants to merge 1 commit into
openclaw:mainfrom
spinnakerbuilding-debug:frank/outbound-thinking-sanitizer
Closed

fix(outbound): sanitize assistant reasoning before delivery#84441
spinnakerbuilding-debug wants to merge 1 commit into
openclaw:mainfrom
spinnakerbuilding-debug:frank/outbound-thinking-sanitizer

Conversation

@spinnakerbuilding-debug

Copy link
Copy Markdown

Summary

  • add an outbound assistant delivery sanitizer that strips private thinking/reasoning tags outside code fences
  • drop reasoning-only assistant payloads before channel delivery
  • apply the sanitizer at durable outbound delivery and reply-dispatcher boundaries while preserving tool-result payloads
  • add regression coverage for sanitizer, durable delivery, and dispatcher paths

Validation

  • npx oxlint src/infra/outbound/assistant-delivery-sanitizer.ts src/infra/outbound/assistant-delivery-sanitizer.test.ts src/infra/outbound/deliver.ts src/infra/outbound/deliver.test.ts src/auto-reply/reply/reply-dispatcher.ts src/auto-reply/reply/reply-dispatcher.sanitizer.test.ts
  • npx vitest run src/infra/outbound/assistant-delivery-sanitizer.test.ts src/infra/outbound/deliver.test.ts src/auto-reply/reply/reply-dispatcher.sanitizer.test.ts — 96 tests passed

Caveat: this is targeted regression coverage; the full repository suite was not run.

@openclaw-barnacle openclaw-barnacle Bot added size: M triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 20, 2026
@clawsweeper

clawsweeper Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

Codex review: found issues before merge. Reviewed June 30, 2026, 4:22 PM ET / 20:22 UTC.

Summary
The PR adds a new assistant-delivery sanitizer, applies it in reply dispatch and durable outbound delivery, and adds focused tests for stripping reasoning before channel send.

PR surface: Source +126, Tests +137. Total +263 across 6 files.

Reproducibility: yes. The remaining broad bug is source-reproducible through channels such as Feishu that still lack a sanitizeText hook, and the Mantis Telegram screenshots show the PR base leaking a Reasoning block before the candidate strips it.

Review metrics: 1 noteworthy metric.

  • Shared sanitizer insertion points: 3 added. The diff adds one dispatcher sanitizer call and two generic durable-delivery sanitizer calls, so the behavior change affects more than one channel or assistant-owned send path.

Stored data model
Persistent data-model change detected: vector/embedding metadata: src/infra/outbound/assistant-delivery-sanitizer.test.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #90684
Summary: This PR is a candidate fix for the broader non-Discord outbound sanitizer issue, but current main has only partial channel-specific fixes and another broader draft PR is not a safe canonical replacement.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦞 diamond lobster ✨ media proof bonus
Patch quality: 🧂 unranked krab
Result: blocked by patch quality or review findings.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Narrow sanitization to assistant-owned/source-aware payloads and preserve Feishu reasoning-preview payloads plus literal direct sends.
  • Replace the new regex sanitizer with the canonical sanitizeAssistantVisibleText path and focused regression tests for known tag/scaffolding variants.
  • Refresh the branch against current main after the code direction is settled.

Risk before merge

  • [P1] Merging this as-is can suppress intended reasoning-preview payloads before channels such as Feishu can render them.
  • [P1] The durable delivery changes can rewrite or drop literal direct-send and plugin-owned payload text because the patch treats shared payloads as assistant-owned.
  • [P1] The new regex sanitizer is a second privacy boundary that misses known canonical sanitizer classes such as namespaced reasoning/tool scaffolding.
  • [P1] The branch is currently conflicting against main, so a refreshed merge result is required before any merge-ready verdict.

Maintainer options:

  1. Choose a source-aware canonical boundary (recommended)
    Gate sanitization to assistant-owned final/block text or carry explicit payload source metadata into durable delivery, and reuse sanitizeAssistantVisibleText while preserving literal sends and reasoning-preview payloads.
  2. Accept universal durable filtering explicitly
    Maintainers could intentionally filter every durable outbound payload, but that needs a compatibility/privacy decision and proof that direct message.send and plugin-owned payloads are acceptable to rewrite or drop.
  3. Defer to the umbrella issue
    Keep this PR open or pause it until Feishu (and non-Discord channels) should apply sanitizeAssistantVisibleText() on outbound text #90684 has an approved shared sanitizer boundary or a safer channel-by-channel plan.

Next step before merge

  • [P2] The remaining blocker is a maintainer boundary and compatibility decision plus a conflicted branch, not a narrow safe automated repair lane.

Security
Needs attention: The diff is privacy-sensitive and introduces an incomplete parallel sanitizer at an outbound delivery boundary.

Review findings

  • [P1] Preserve reasoning-preview payloads — src/auto-reply/reply/reply-dispatcher.ts:176-181
  • [P1] Preserve caller-owned durable payloads — src/infra/outbound/deliver.ts:790-792
  • [P1] Reuse the canonical visible-text sanitizer — src/infra/outbound/assistant-delivery-sanitizer.ts:18-26
Review details

Best possible solution:

Use the canonical sanitizeAssistantVisibleText at a source-aware assistant-owned boundary, or finish per-channel sanitizeText coverage, while preserving reasoning-preview lanes and literal caller-owned sends.

Do we have a high-confidence way to reproduce the issue?

Yes. The remaining broad bug is source-reproducible through channels such as Feishu that still lack a sanitizeText hook, and the Mantis Telegram screenshots show the PR base leaking a Reasoning block before the candidate strips it.

Is this the best way to solve the issue?

No. This is a plausible mitigation, but the best fix should reuse the canonical sanitizer at a source-aware assistant-owned boundary or complete channel-level hooks instead of treating all generic payloads as assistant text.

Full review comments:

  • [P1] Preserve reasoning-preview payloads — src/auto-reply/reply/reply-dispatcher.ts:176-181
    This dispatcher path is shared by channel deliverers that intentionally render payload.isReasoning, such as Feishu's formatReasoningMessage path. Because the new sanitizer is called without a reasoning-enabled option, isReasoning block/final payloads return null before those channels can display their configured reasoning preview.
    Confidence: 0.92
  • [P1] Preserve caller-owned durable payloads — src/infra/outbound/deliver.ts:790-792
    The durable delivery normalizer handles direct message.send and plugin-owned payloads, not only assistant replies. Treating every normalized payload as role assistant can drop or rewrite legitimate literal Reasoning: content or reasoning-tag documentation before it reaches the channel.
    Confidence: 0.9
  • [P1] Reuse the canonical visible-text sanitizer — src/infra/outbound/assistant-delivery-sanitizer.ts:18-26
    The new regex helper duplicates behavior already covered by sanitizeAssistantVisibleText and stripReasoningTagsFromText, but misses known namespaced tags and broader internal scaffolding. At this privacy-sensitive boundary, that drift can leave known leak formats uncovered.
    Confidence: 0.87

Overall correctness: patch is incorrect
Overall confidence: 0.92

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 640258d7b31d.

Label changes

Label justifications:

  • P1: The PR addresses private reasoning/tool-trace leakage in user-visible channel delivery, but the current patch can also break outbound message delivery.
  • merge-risk: 🚨 compatibility: The diff changes default durable outbound text handling for existing direct and plugin-owned payloads without source metadata or upgrade proof.
  • merge-risk: 🚨 message-delivery: The diff can suppress reasoning previews and drop intended outbound messages before channel send by treating shared payloads as assistant reasoning.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦞 diamond lobster and patch quality is 🧂 unranked krab.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (screenshot): Mantis Telegram Desktop screenshots show after-fix real behavior: the baseline displays the full Reasoning/Final answer block and the candidate displays only the visible final text.
  • proof: sufficient: Contributor real behavior proof is sufficient. Mantis Telegram Desktop screenshots show after-fix real behavior: the baseline displays the full Reasoning/Final answer block and the candidate displays only the visible final text.
  • proof: 📸 screenshot: Contributor real behavior proof includes screenshot evidence. Mantis Telegram Desktop screenshots show after-fix real behavior: the baseline displays the full Reasoning/Final answer block and the candidate displays only the visible final text.
  • mantis: telegram-visible-proof: Mantis should capture Telegram visible proof. The PR changes visible Telegram chat delivery and the existing Telegram Desktop proof lane directly demonstrates that behavior.
Evidence reviewed

PR surface:

Source +126, Tests +137. Total +263 across 6 files.

View PR surface stats
Area Files Added Removed Net
Source 3 132 6 +126
Tests 3 137 0 +137
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 6 269 6 +263

Security concerns:

  • [medium] Incomplete sanitizer misses known private formats — src/infra/outbound/assistant-delivery-sanitizer.ts:18
    The new sanitizer only lists unprefixed reasoning tag names, while current shared helpers strip known namespaced forms such as antml/mm and antthinking, along with broader internal scaffolding classes. If this becomes the privacy boundary, known private-reasoning or tool-call formats can still leak.
    Confidence: 0.82

What I checked:

  • Repository policy applied: Root AGENTS.md and the scoped outbound guide were read; their compatibility and shared-outbound review guidance applies because this PR changes generic delivery behavior. (AGENTS.md:31, 640258d7b31d)
  • PR filters non-tool dispatcher payloads before channel delivery: The PR sanitizes every non-tool normalized reply payload as assistant text and returns false when the sanitizer drops it. (src/auto-reply/reply/reply-dispatcher.ts:176, badf3f8cabcf)
  • Feishu has an intentional reasoning-preview delivery path: Current main formats payloads with isReasoning before sending, so dispatcher-level dropping would bypass a channel feature that expects to see those payloads. (extensions/feishu/src/reply-dispatcher.ts:655, 640258d7b31d)
  • PR treats generic durable outbound payloads as assistant-owned: The PR inserts sanitizeAssistantForDelivery after channel normalization in the durable delivery path, which also handles direct message sends and plugin-owned payloads. (src/infra/outbound/deliver.ts:790, badf3f8cabcf)
  • Direct sends feed caller-owned text into durable delivery: sendMessage builds outbound payloads directly from params.payloads or params.content, so literal caller text can reach the generic delivery normalizer. (src/infra/outbound/message.ts:315, 640258d7b31d)
  • Canonical sanitizer already exists: Current main's sanitizeAssistantVisibleText delivery profile strips reasoning tags plus model tokens, tool-call XML, internal trace lines, legacy blocks, and downgraded tool-call text. (src/shared/text/assistant-visible-text.ts:921, 640258d7b31d)

Likely related people:

  • steipete: Introduced and refactored the canonical assistant-visible sanitizer and split shared outbound delivery flow that this PR bypasses or changes. (role: feature owner / adjacent sanitizer owner; confidence: high; commits: 712479eea1a1, c63a4f0f13fc, 1d0a4d1be2da; files: src/shared/text/assistant-visible-text.ts, src/infra/outbound/deliver.ts)
  • maweibin: Recent current-main work extended the visible-text sanitizer for namespaced tool-call XML, directly overlapping the PR's new parallel sanitizer surface. (role: recent sanitizer contributor; confidence: medium; commits: b2787a1c7a7a; files: src/shared/text/assistant-visible-text.ts)
  • jailbirt: Authored the merged Google Chat slice for the same outbound internal-trace leak class using the canonical sanitizeAssistantVisibleText hook. (role: adjacent channel sanitizer contributor; confidence: medium; commits: 696c62400803; files: extensions/googlechat/src/channel.adapters.ts, extensions/googlechat/src/channel.test.ts)
  • mushuiyu886: Authored the current-main Telegram outbound sanitizer hook that overlaps this PR's Telegram proof surface. (role: recent Telegram contributor; confidence: medium; commits: 434d752dd60c; files: extensions/telegram/src/outbound-adapter.ts)
  • obviyus: Recent current-main Feishu commits touch the channel area still affected by the broader sanitizer gap. (role: recent Feishu area contributor; confidence: medium; commits: f078962d1759; files: extensions/feishu/src/outbound.ts, extensions/feishu/src/send.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. labels May 20, 2026
@clawsweeper

clawsweeper Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?
  • The egg game starts only after the PR passes the real-behavior proof check.
  • Before that, no creature or rarity is rolled. The treat waits for real proof.
  • This is still just collectible flavor: proof affects review readiness, not creature quality.

@clawsweeper clawsweeper Bot added the merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. label Jun 19, 2026
@clawsweeper clawsweeper Bot added the mantis: telegram-visible-proof Mantis should capture Telegram visible proof. label Jun 20, 2026
@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 20, 2026
@clawsweeper
clawsweeper Bot temporarily deployed to qa-live-shared June 20, 2026 02:49 Inactive
@openclaw-mantis

Copy link
Copy Markdown
Contributor

Mantis Telegram Desktop Proof

Summary: Mantis captured native Telegram Desktop before/after GIFs showing Telegram message delivery changed from the full literal Reasoning block to only the visible final text.

Main screenshot This PR screenshot
Baseline native Telegram Desktop screenshot Candidate native Telegram Desktop screenshot
Main This PR
Baseline native Telegram Desktop proof GIF Candidate native Telegram Desktop proof GIF

Motion-trimmed clips:

Raw QA files: https://artifacts.openclaw.ai/mantis/telegram-desktop/pr-84441/run-27857897022-1/index.json

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 21, 2026
@openclaw-barnacle

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jul 15, 2026
@openclaw-barnacle

Copy link
Copy Markdown

Closing due to inactivity.
If you believe this PR should be revived, post in #clawtributors on Discord to talk to a maintainer.
That channel is the escape hatch for high-quality PRs that get auto-closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mantis: telegram-visible-proof Mantis should capture Telegram visible proof. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P1 High-priority user-facing bug, regression, or broken workflow. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: M stale Marked as stale due to inactivity status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant