-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
Filter noisy invalid_request_error subtypes (e.g. thinking-signature) out of user-facing assistant text #87475
Copy link
Copy link
Closed
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.This issue has meaningful maintainer-visible impact outside the owned taxonomy.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Description
Metadata
Metadata
Assignees
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.This issue has meaningful maintainer-visible impact outside the owned taxonomy.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Type
Fields
Priority
None yet
Summary
formatAssistantErrorTextinsrc/agents/embedded-agent-helpers/errors.tsreturnsLLM request rejected: ${raw_message}verbatim for any providerinvalid_request_error. For most cases (malformed JSON, image-size, validation diagnostics) this is the correct call — the raw provider message has diagnostic value, and the existing testkeeps non-streaming provider request-validation syntax diagnosticsatsrc/agents/embedded-agent-helpers/errors.test.ts:48-55documents that intent.But some
invalid_request_errorsubtypes produce text that is noise to end users and harmful in delivered channels. The clearest example is Anthropic's thinking-block signature rejection, e.g.:This was a real production leak to a Telegram channel before the recent recovery wiring landed.
Now that
wrapAnthropicStreamWithRecoveryis wired in (#86886 /66965f5008) the loop is gone, so this should be rare. But the same text can still reach the channel if the wrap's single-retry doesn't succeed, or for otherinvalid_request_errorsubtypes that are similarly opaque to a user (e.g. internal validation strings that namemessages.N.content.Mpaths).Filing this as a product-direction question rather than a fix-it PR because the existing behavior is deliberate and tested — changing it crosses an owner-decision line.
Real-world observation
Two separate production occurrences on a Mac-mini Telegram-bot install (channel
dev,2026.5.22at the time):messages.13.content.19: Invalid \signature` in `thinking` block` delivered as assistant text to the channel.messages.3.content.19: Invalid \signature` in `thinking` block`, three consecutive messages, before the session-reset safety net broke the cycle and discarded conversation history.In both cases the bot's user saw a string of API-shaped error text where they were expecting a normal reply. The session-reset that eventually fired also lost the in-conversation memory of an in-progress task (PDF iteration with several deliverables).
Suggested direction (not prescriptive)
Two possible shapes — happy to defer to maintainers on which they prefer or whether something else is better:
A. Filter known-noisy invalid_request_error subtypes to a generic user-facing message ("I had trouble with that request — try again or
/newfor a fresh session") while preserving the raw text in operator logs. Specifically, the signature-block class of error matches the sameTHINKING_BLOCK_ERROR_PATTERNintroduced for recovery inembedded-agent-runner/thinking.ts; that regex could double as the "this is noise to users" detector here.B. Default to generic for any invalid_request_error whose raw text contains
messages.N.content.Mpath references — those are API-internal path notation and never user-actionable. Keep the raw passthrough for cleaner messages (e.g. JSON validation that names a field but not an API path).If neither feels right, this can just be closed — the upstream wiring fix already removed the loop, which was the worst part of the symptom. The remaining leak surface is small.
What I am not proposing
invalid_request_errorcases (malformed JSON, etc.) — those keep current behavior under either A or B.wrapAnthropicStreamWithRecovery— that work is done.Environment
2026.5.22(at the time of the production occurrence), now updated to2026.5.27on the same installclaude-sonnet-4-6