Skip to content

fix: scope error-detection heuristics to error source in sanitizeUserFacingText#12052

Closed
skylarkoo7 wants to merge 2 commits intoopenclaw:mainfrom
skylarkoo7:fix/12022-billing-false-positive
Closed

fix: scope error-detection heuristics to error source in sanitizeUserFacingText#12052
skylarkoo7 wants to merge 2 commits intoopenclaw:mainfrom
skylarkoo7:fix/12022-billing-false-positive

Conversation

@skylarkoo7
Copy link

@skylarkoo7 skylarkoo7 commented Feb 8, 2026

Summary

sanitizeUserFacingText() applies error-detection heuristics (HTTP status matching, error-prefix detection, rate-limit rewriting) to all outbound text, including normal assistant responses. This causes false positives when the assistant's reply contains billing keywords ("credit balance", "402"), HTTP-status-like prefixes, or error-related phrases — the entire response gets replaced with a generic error message.

This PR adds an optional source parameter ("assistant" | "error") so callers can indicate the origin of the text:

  • "assistant" — skips the error-detection heuristics; only applies safe transforms (tag stripping, duplicate block collapse)
  • "error" — applies the full error-detection and rewriting logic (current behavior)
  • omitted — backward-compatible, behaves as before

All callers that process normal assistant content now pass { source: "assistant" }:

File Role
src/auto-reply/reply/normalize-reply.ts Outbound reply normalization for all channels
src/auto-reply/reply/agent-runner-execution.ts Streaming text normalization during model execution
src/agents/pi-embedded-utils.ts Assistant text extraction from message blocks
src/agents/tools/sessions-helpers.ts Session transcript text extraction

Error-path callers (e.g. formatAssistantErrorText) continue to use the default behavior and apply full error detection.

Test plan

  • Existing unit tests pass (18/18 across sanitizeUserFacingText, isBillingErrorMessage, formatAssistantErrorText, and classifyFailoverReason suites)
  • Verified that assistant responses containing "credit balance", "402", "billing upgrade" are no longer mangled when delivered through the outbound reply pipeline
  • Error messages (actual API failures) continue to be formatted correctly

Fixes #12022

Greptile Overview

Greptile Summary

This PR scopes sanitizeUserFacingText()’s error-detection/rewriting heuristics (HTTP status formatting, error-prefix detection, rate-limit/overload rewriting, billing keyword detection) to text that is known to originate from an error source.

It introduces an optional source option ("assistant" | "error", defaulting to the previous behavior when omitted) and updates outbound/assistant-text call sites (normalize-reply, streaming normalization, transcript extraction) to pass { source: "assistant" } so normal assistant replies don’t get replaced by generic error messages when they contain billing keywords (e.g. “credit balance”) or status-code-like text.

Confidence Score: 4/5

  • This PR looks safe to merge, with the main remaining risk being insufficient test coverage for the new { source: "assistant" } behavior.
  • Behavioral change is narrowly scoped and call sites are consistently updated to pass source: "assistant" for normal assistant output while keeping backward-compatible defaults for error paths. The primary concern is that the test suite doesn’t exercise the new code path, so the intended prevention of false positives (billing keywords/status-like text) and preservation of safe transforms in assistant mode aren’t locked in by tests.
  • src/agents/pi-embedded-helpers/errors.ts (add/adjust unit tests to cover source: "assistant" and optionally source: "error")

(3/5) Reply to the agent's comments like "Can you suggest a fix for this @greptileai?" or ask follow-up questions!

…ror source

sanitizeUserFacingText() applies error-detection heuristics (HTTP status
matching, error-prefix detection, rate-limit / overload rewriting) to ALL
outbound text, including normal assistant responses.  This causes false
positives when the assistant's reply happens to contain billing keywords
("credit balance", "402"), HTTP-like prefixes, or error-related phrases.

Add an optional `source` parameter ("assistant" | "error") so callers
can indicate the origin of the text.  When source is "assistant", skip
the error-detection checks and only apply safe transforms (tag stripping,
duplicate block collapse).  Existing callers without the parameter keep
the current behavior for backward compatibility.

Update all callers that process normal assistant content to pass
`{ source: "assistant" }`:
- normalize-reply.ts (outbound reply normalization)
- agent-runner-execution.ts (streaming text normalization)
- pi-embedded-utils.ts (assistant text extraction)
- sessions-helpers.ts (session text extraction)

Fixes openclaw#12022
@openclaw-barnacle openclaw-barnacle bot added the agents Agent runtime and tooling label Feb 8, 2026
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Add 8 tests verifying the { source: "assistant" } code path:
- billing keywords (credit balance, billing upgrade, insufficient credits)
  are NOT rewritten
- HTTP status codes (500, 402) are NOT rewritten
- error-prefixed text ("Error: ...", "Failed: ...") is NOT rewritten
- raw JSON error payloads are NOT rewritten
- <final> tag stripping still works
- duplicate paragraph collapse still works
- normal multi-line assistant content passes through unchanged
@Takhoffman
Copy link
Contributor

Fixed in #12988.

This will go out in the next OpenClaw release.

If you still see this after updating to the first release that includes #12988, please open a new issue with:

  • your OpenClaw version
  • channel (Telegram/Slack/etc)
  • the exact prompt/response that got rewritten
  • whether Web UI showed the full text vs the channel being rewritten
  • relevant logs around send/normalize (if available)

Link back here for context.

@Takhoffman
Copy link
Contributor

Closing as superseded by the merged sanitize/error-context work:

This PR’s intent appears covered by those merged changes and current mainline tests.

@Takhoffman Takhoffman closed this Feb 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

False positive billing error detection on assistant response text

2 participants

Comments