Skip to content

[RFC] Rate-limit fallback: user-visible error + immediate switch notification (message-lifecycle Phase 2 extension) #92672

Description

@kumaxs

Summary

当 primary 模型触发 rate_limit / billing / overloaded 时,OpenClaw 静默挂起 session 30 分钟,用户只感到"卡住",看到的是 didSendViaMessagingTool=false + non_deliverable_terminal_turn,零提示。

本提案让 OpenClaw 在 fallback 触发时自动通过 channel 发送一条人话消息给用户,并在现有基础上保持 fallback 链立即尝试、context 自动透传。改动量 ~65 行纯增量,零删除。

Background

The Gap

FailoverError 在 embedded-agent.ts:2691 已经有人话消息

throw new FailoverError(
  "The AI service is temporarily rate-limited. Please try again in a moment.",
  { reason: "rate_limit", provider: "minimax", model: "MiniMax-M3", suspend: true }
);

但这个消息只存在 error 对象里,从未被投递到 channel

同时,message-lifecycle-refactor.md Phase 2 已经设计好了:

  • MessageOrigin.kind === "gateway_failure" (已有)
  • core.messages.send 完整 lifecycle:begin → render → send → commit → fail
  • DeliveryFailureKind 已将 rate_limit 分类为可重试
  • shouldDropOpenClawEcho 防 bot room 自循环

缺的只是一根线:把这根线从 FailoverError throw 点接到 core.messages.send

Proposed Changes

4 files, ~65 lines, pure additive

File Change Lines
docs/concepts/message-lifecycle-refactor.md +3 MessageOrigin.gateway_failure code enums +10
src/agents/embedded-agent.ts 🆕 notifyFailoverToUser() + 2 call sites +50
src/agents/failover-error.ts + userMessage? optional field +3
src/agents/model-fallback.ts 1 throw site + userMessage +2

Design

FailoverError thrown
  → notifyFailoverToUser({ session, failoverError })
    → humanizeFailoverError(err) → "⚠️ 主模型 MiniMax-M3 触发速率限制..."
    → core.messages.send(MessageOrigin.kind="gateway_failure", best_effort)
  → if (err.suspend) suspendSession(...)

Two call sites to instrument:

  • embedded-agent.ts ~L3503: after prompt submission failover
  • embedded-agent.ts ~L3827: after assistant call failover

New MessageOrigin codes

"model_rate_limited"      // rate_limit
"model_overloaded"        // overloaded
"model_billing_exhausted" // billing

FailoverError extension

class FailoverError extends Error {
  // ...existing fields...
  userMessage?: string;  // NEW: pre-constructed user-facing message
}

notifyFailoverToUser() prefers userMessage if set; falls back to auto-translating failoverError.reason.

Humanized messages (examples)

⚠️ 主模型 MiniMax-M3 触发速率限制(rate_limit),约 30 分钟后自动恢复。
⚠️ 主模型 MiniMax-M3 暂时过载,正在切换备用模型。
⚠️ 主模型 MiniMax-M3 账户配额用尽,约 30 分钟后自动恢复。

What This Does NOT Change

  • ❌ Does NOT modify fallback chain logic(candidates[i+1] already tries immediately)
  • ❌ Does NOT change suspendSession 30min wait
  • ❌ Does NOT break session reuse(runEmbeddedAgent context passthrough already works)
  • ❌ Does NOT modify model resolution or configuration
  • ❌ No tests(per decision: avoid polluting openclaw environment)

Key Insight

The foundation already exists — both sides:

  • FailoverError side:already has human-readable message string at throw site
  • Message delivery sidecore.messages.send already has full lifecycle + MessageOrigin.gateway_failure type

We're not building new infrastructure — we're connecting two things that already exist but aren't wired together.

Compatibility & Risk

Risk Mitigation
User overwhelmed by error messages userMessage only constructed for rate_limit/billing/overloaded
Channel doesn't support outbound best_effort durability → silent failure
Modifies Phase 2 design Pure additive: new enums + new hook call; existing schema unchanged
Emoji preference i18n framework ready for configurable icon set

Code References

  • dist/failover-error-CmbjIKMV.js L24-61: FailoverError class + reason→status mapping
  • dist/embedded-agent-CpxCz9I4.js L2691: throw FailoverError with human message
  • dist/embedded-agent-CpxCz9I4.js L1781: fallback_model decision throw
  • dist/embedded-agent-CpxCz9I4.js L3503, L3827: suspendSession call sites
  • dist/model-fallback-D14rZfHX.js L1185-L1452: fallback chain nextCandidate
  • dist/model-fallback-D14rZfHX.js L466: suspendSession implementation
  • dist/delivery.runtime-ChtITtgt.js L83: didSendViaMessagingTool passthrough
  • docs/concepts/message-lifecycle-refactor.md: Phase 2 durable send core design

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions