-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
[RFC] Rate-limit fallback: user-visible error + immediate switch notification (message-lifecycle Phase 2 extension) #92672
Copy link
Copy link
Open
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.Auth, provider routing, model choice, or SecretRef resolution may break.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.Channel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Description
Metadata
Metadata
Assignees
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.Auth, provider routing, model choice, or SecretRef resolution may break.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.Channel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Type
Fields
Priority
None yet
Summary
当 primary 模型触发 rate_limit / billing / overloaded 时,OpenClaw 静默挂起 session 30 分钟,用户只感到"卡住",看到的是
didSendViaMessagingTool=false+non_deliverable_terminal_turn,零提示。本提案让 OpenClaw 在 fallback 触发时自动通过 channel 发送一条人话消息给用户,并在现有基础上保持 fallback 链立即尝试、context 自动透传。改动量 ~65 行纯增量,零删除。
Background
/modelswitch + manual gateway restart don't reset provider cooldown; LLM idle timeout is mis-classified as a transient failure and poisons the fallback chain for the whole gateway process lifetime #91659:Cooldown set bugbad20dda—— M3 rate_limit_error (2077500/2077500),用户发"现在是哪个模型",零回复零提示The Gap
FailoverError 在
embedded-agent.ts:2691已经有人话消息:但这个消息只存在 error 对象里,从未被投递到 channel。
同时,
message-lifecycle-refactor.mdPhase 2 已经设计好了:MessageOrigin.kind === "gateway_failure"(已有)core.messages.send完整 lifecycle:begin → render → send → commit → failDeliveryFailureKind已将rate_limit分类为可重试shouldDropOpenClawEcho防 bot room 自循环缺的只是一根线:把这根线从 FailoverError throw 点接到
core.messages.send。Proposed Changes
4 files, ~65 lines, pure additive
docs/concepts/message-lifecycle-refactor.mdMessageOrigin.gateway_failurecode enumssrc/agents/embedded-agent.tsnotifyFailoverToUser()+ 2 call sitessrc/agents/failover-error.tsuserMessage?optional fieldsrc/agents/model-fallback.tsDesign
Two call sites to instrument:
embedded-agent.ts ~L3503: after prompt submission failoverembedded-agent.ts ~L3827: after assistant call failoverNew MessageOrigin codes
FailoverError extension
notifyFailoverToUser()prefersuserMessageif set; falls back to auto-translatingfailoverError.reason.Humanized messages (examples)
What This Does NOT Change
candidates[i+1]already tries immediately)suspendSession30min waitrunEmbeddedAgentcontext passthrough already works)Key Insight
The foundation already exists — both sides:
core.messages.sendalready has full lifecycle +MessageOrigin.gateway_failuretypeWe're not building new infrastructure — we're connecting two things that already exist but aren't wired together.
Compatibility & Risk
userMessageonly constructed for rate_limit/billing/overloadedbest_effortdurability → silent failureCode References
dist/failover-error-CmbjIKMV.jsL24-61: FailoverError class + reason→status mappingdist/embedded-agent-CpxCz9I4.jsL2691: throw FailoverError with human messagedist/embedded-agent-CpxCz9I4.jsL1781: fallback_model decision throwdist/embedded-agent-CpxCz9I4.jsL3503, L3827: suspendSession call sitesdist/model-fallback-D14rZfHX.jsL1185-L1452: fallback chain nextCandidatedist/model-fallback-D14rZfHX.jsL466: suspendSession implementationdist/delivery.runtime-ChtITtgt.jsL83: didSendViaMessagingTool passthroughdocs/concepts/message-lifecycle-refactor.md: Phase 2 durable send core design