fix(agent): prevent sticky model fallback#75270
Conversation
|
Codex review: found issues before merge. Reviewed July 3, 2026, 11:49 PM ET / 03:49 UTC. Summary PR surface: Source +617, Tests +1089, Docs 0. Total +1706 across 15 files. Reproducibility: yes. for the behavior this PR changes: current main and v2026.6.11 persist auto fallback state and write actual provider/model usage into the session, while the PR proof demonstrates non-sticky after-fix behavior with a local endpoint. The remaining question is whether that should be the default or an opt-in product mode. Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review findings
Review detailsBest possible solution: Preserve sticky fallback as the default, decide optional per-turn fallback in #80502, and keep #92819 scoped to the stale-origin bug. Do we have a high-confidence way to reproduce the issue? Yes for the behavior this PR changes: current main and v2026.6.11 persist auto fallback state and write actual provider/model usage into the session, while the PR proof demonstrates non-sticky after-fix behavior with a local endpoint. The remaining question is whether that should be the default or an opt-in product mode. Is this the best way to solve the issue? No, not as-is. The narrower maintainable path is to preserve sticky fallback by default, add any per-turn behavior only behind a maintainer-approved config contract, and leave the stale-origin bug to #92819. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 9d68f877ac3e. Label changesLabel justifications:
Evidence reviewedPR surface: Source +617, Tests +1089, Docs 0. Total +1706 across 15 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
08e966c to
d3789d8
Compare
d3789d8 to
715de76
Compare
|
Addressed the ClawSweeper follow-up on the current head:
|
51c6e44 to
bee2196
Compare
openclaw-barnacle
Bot
added
the
triage: needs-real-behavior-proof
bee2196 to
9e24869
Compare
9e24869 to
cf59a66
Compare
cf59a66 to
62d6741
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 62d67410f9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
62d6741 to
a60346b
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 50d070ca82
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
e977025 to
85e5029
Compare
6325382 to
3df47b1
Compare
327d20f to
60474b9
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 60474b9601
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
825ae6d to
6281bf2
Compare
208a95d to
f6ea2bc
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f6ea2bce43
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
99985d6 to
1d3df34
Compare
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
1d3df34 to
aae1deb
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aae1deb4ac
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| runResult.meta?.agentMeta?.provider ?? fallbackProvider ?? queued.run.provider; | ||
| const fallbackStateEntry = | ||
| (replySessionKey ? sessionStore?.[replySessionKey] : undefined) ?? activeSessionEntry; | ||
| const autoFallbackSelectionToClear = snapshotAutoFallbackSelection(fallbackStateEntry); |
There was a problem hiding this comment.
When queued.run.inputProvenance is an inter-session/tool-mediated source, preserveUserFacingSessionState is true and this runner already preserves model persistence and fallback notices, but this snapshot is still taken unconditionally and later drives cleanup at if (autoFallbackSelectionToClear && didPersistRunSessionUsage). In that preserved-followup scenario, a successful background followup can clear the session's auto fallback override/status even though user-facing session state is supposed to remain unchanged; fresh evidence beyond the prior fallback-notice comment is that the new cleanup snapshot is not guarded by !preserveUserFacingSessionState.
Useful? React with 👍 / 👎.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
This pull request has been automatically marked as stale due to inactivity. |
clawsweeper
Bot
added
rating: 🦪 silver shellfish
clawsweeper
Bot
added
rating: 🧂 unranked krab
|
This pull request has been automatically marked as stale due to inactivity. |
|
ClawSweeper status: review started. I am starting a fresh review of this pull request: fix(agent): prevent sticky model fallback This is item 1/1 in the current shard. Shard 4/22. This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking. Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted. |
2 participants
Summary
Root cause
Successful fallback candidates were persisted into session override state but only rolled back on errors/classified failures. Usage persistence also wrote
providerUsed/modelUsedinto the session identity, so one fallback run could pin later turns to the smaller fallback model even after the primary model recovered.Validation
node scripts/test-projects.mjs src/auto-reply/reply/agent-runner-execution.test.ts src/auto-reply/reply/session.test.ts src/auto-reply/reply/followup-runner.test.tsReal behavior proof
Behavior or issue addressed: A temporary model fallback should not make the session stick to the fallback model. After a primary-model rate limit sends one turn to a configured fallback, the next turn in the same session should try the configured primary again while usage/status still records the actual fallback runtime used for the fallback turn.
Real environment tested: Local source checkout at PR head
c4f5337a7fdcc9f11c17ff6f42d8e8295fb77eed, Nodev24.15.0, pnpm11.1.0, isolatedOPENCLAW_HOME=/var/tmp/openclaw-pr75270-proof/home, isolated workspace/var/tmp/openclaw-pr75270-proof/workspace, and a local OpenAI-compatible HTTP endpoint on127.0.0.1. No real user session, live channel, or production OpenClaw home was used.Exact steps or command run after this patch: 1. Configured
agents.defaults.model.primaryasprimary-local/primary-modelwithfallbacks: ["fallback-local/fallback-model"].2. Started a local OpenAI-compatible
/v1/chat/completionsserver that returns four HTTP 429 responses forprimary-model, then returns streaming success forfallback-model; after that,primary-modelreturns streaming success.3. Ran two local embedded agent turns against the same session id:
/usr/bin/node openclaw.mjs agent --local --agent main --session-id pr75270-proof --thinking off --json --timeout 120 --message <proof message>Evidence after fix: Copied terminal output from
/var/tmp/openclaw-pr75270-proof/proof-output.json:{ "first": { "exitCode": 0, "text": "PR75270 fallback proof OK", "provider": "fallback-local", "model": "fallback-model", "fallbackAttempts": [ { "provider": "primary-local", "model": "primary-model", "error": "429 proof primary rate limit", "reason": "rate_limit", "status": 429 } ], "requestModels": [ "primary-model", "primary-model", "primary-model", "primary-model", "fallback-model", "fallback-model" ], "storeAfterTurn": { "modelProvider": "fallback-local", "model": "fallback-model", "providerOverride": null, "modelOverride": null, "modelOverrideSource": null, "fallbackNoticeSelectedModel": null, "fallbackNoticeActiveModel": null } }, "second": { "exitCode": 0, "text": "PR75270 primary recovered OK", "provider": "primary-local", "model": "primary-model", "fallbackAttempts": [], "requestModels": [ "primary-model", "primary-model" ], "storeAfterTurn": { "modelProvider": "primary-local", "model": "primary-model", "providerOverride": null, "modelOverride": null, "modelOverrideSource": null, "fallbackNoticeSelectedModel": null, "fallbackNoticeActiveModel": null } }, "allRequestModels": [ "primary-model", "primary-model", "primary-model", "primary-model", "fallback-model", "fallback-model", "primary-model", "primary-model" ] }The runtime log also showed the fallback decision on the first turn:
Observed result after fix: The first turn used the configured fallback after the synthetic primary returned 429, and
fallbackAttemptsrecorded the primary failure. The same session's persisted auto override fields were clear after the fallback turn. The second turn, with the primary endpoint recovered, requested and completed onprimary-local/primary-modelwith no fallback attempts. This verifies the session did not stay pinned to the fallback model.What was not tested: This proof used a local OpenAI-compatible endpoint rather than a paid hosted provider, did not exercise real Telegram/WhatsApp delivery, and did not test provider-specific OAuth rotation. Those paths are covered separately by the automated tests listed above.