-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
Add optional per-turn model fallback without persisting session override #80502
Copy link
Copy link
Open
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.Auth, provider routing, model choice, or SecretRef resolution may break.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🌊 off-meta tidepoolIssue quality rating does not apply to this item.Issue quality rating does not apply to this item.
Description
Metadata
Metadata
Assignees
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.Auth, provider routing, model choice, or SecretRef resolution may break.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🌊 off-meta tidepoolIssue quality rating does not apply to this item.Issue quality rating does not apply to this item.
Type
Fields
Priority
None yet
Summary
Add an optional mode for model fallbacks to be scoped to a single turn instead of persisting the fallback model as the active session override.
Problem
Model fallbacks are useful when a primary model is temporarily unavailable, rate-limited, or over capacity. However, when a fallback candidate succeeds, the runtime can persist that fallback provider/model into the session state as the active model selection.
That is useful for some workflows, but it can be surprising for agents that should prefer a high-quality primary model whenever it is available:
For operational bots, the desired behavior is often: keep the reply alive with a fallback for this one turn, but try the configured primary again on the next user turn.
Proposal
Support a configurable fallback persistence policy, for example:
{ "agents": { "list": [ { "id": "example-agent", "model": { "primary": "openai-codex/gpt-5.5", "fallbacks": ["zai/glm-5-turbo"], "fallbackPersistence": "turn" } } ] } }Possible modes:
session— current behavior / durable fallback selection where appropriate.turn— fallback is used only for the current run; after a successful fallback turn, restore the prior session model state so the next turn starts from the configured/default primary again.The default should preserve existing behavior for compatibility.
Expected behavior for
turnNotes from local experiment
A local experiment implemented this by rolling back the temporary fallback candidate selection after
runWithModelFallbackreturns a successful fallback result. The relevant area wassrc/auto-reply/reply/agent-runner-execution.ts, around the existingpendingFallbackCandidateRollbackhandling.A regression test asserted that after a successful fallback turn,
providerOverride,modelOverride, andmodelOverrideSourceare not left set to the fallback model.This issue is intended to discuss making that behavior configurable rather than changing the global default unconditionally.