You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Session model pinning persists indefinitely after fallback, never reconciles with configured primary — confirmed across two independent occurrences (v2026.4.15 and v2026.5.6) #82544
Note for maintainers: this is an update to the case originally documented on 24 April 2026. The bug has now recurred in a different OpenClaw version with a different fallback model and an additional anomalous state (provider: null). The workaround is identical and remains the only known mitigation. Posting this as a fresh, comprehensive write-up so the upstream record is complete.
Bug type
Behavior bug (incorrect output/state without crash). State-persistence regression.
Summary
When a model fallback occurs during a turn (primary temporarily unavailable due to rate limit, billing, or timeout), the fallback model is persisted in the session state stored in sessions.json. Once persisted, that session uses the pinned model for every future turn indefinitely, with no mechanism to retry the configured primary even after the primary has fully recovered.
This is the session-store counterpart to #47705 (same pathology at the openclaw.json agent-config level). The underlying defect is the same: a model selection made under temporary failure conditions becomes permanent and silently overrides user configuration.
Confirmed on two separate occurrences, three weeks apart, across two OpenClaw releases. Same single-user environment, same agent, same fallback chain. Different pinned model each time.
Every inbound Slack message read this state, skipped Haiku entirely, and started the turn on Gemini. Gateway log evidence:
embedded run failover decision: runId=... stage=assistant
decision=fallback_model reason=timeout
from=google/gemini-3.1-flash-lite-preview profile=sha256:0741309971a0
The from=google/gemini-... value confirms Gemini was the starting model for the turn, not a fallback from Haiku.
WhatsApp sessions never accumulated the pinning because that channel's session key is separate and had been created under healthier conditions.
Occurrence 2 — 16 May 2026 (v2026.5.6)
Symptom
Slack DM error message visible to the user:
"I couldn't reach the configured model backend google/gemini-3.1-flash-lite-preview. Fallback used xai/grok-4-1-fast-non-reasoning, but it produced no visible reply."
State observed
After three weeks of normal operation and no manual changes to sessions.json, the Slack DM session had been re-pinned. This time the pinned state was different from occurrence 1:
The pinned model is the second fallback in the chain (Grok), not the first (Gemini). This indicates the pinning logic captures whichever model successfully completes a turn, not specifically the first fallback.
provider is null — schema-anomalous (should be xai). The session entry was written with an incomplete provider attribution.
Interpretation of the user-facing error
Reading the error message together with the persisted state suggests the following sequence:
The session was previously pinned to Gemini (state from a prior failover).
A new inbound message read that state and attempted Gemini directly.
Gemini failed (timeout or transient unavailability).
The runtime fell through to Grok per the configured fallback chain.
Grok returned an empty / non-visible response.
Despite the empty response, the runtime persisted Grok as the new pinned model (overwriting Gemini), with provider field unset.
The persistence path treats "model that completed the turn at the API level" as success even when the response body is empty. This is an independent secondary issue worth flagging (see "Related secondary issue" below).
Files associated with the pinned session in v2026.5.6
Occurrence 2 surfaced additional per-session files that were not present (or not noted) in occurrence 1:
The workaround below moves only the transcript .jsonl aside (as in April). The trajectory files were not touched and produced no observable post-restart issue. Maintainers may want to confirm whether trajectory files should also be archived as part of any future reset procedure.
Why automatic recovery never happens
By the time either occurrence was diagnosed, the configured primary (Haiku) was fully operational — verified by:
Direct Anthropic API test from the same host.
Concurrent traffic on WhatsApp succeeding with Haiku.
But the Slack session never retried Haiku. There is no periodic reconciliation along the lines of:
"My config says primary=Haiku. My session state says model=X. Is Haiku healthy now? If yes, retry Haiku and update session state on success."
The session holds the pinned model forever, until manually cleared at the file-system level.
The docs at /concepts/model-failover describe primary-probing behavior at the provider-cooldown level ("The primary candidate may be probed near cooldown expiry"). That mechanism does not apply to session-level pinned overrides.
Workaround (reversible, production-safe, reproducible across both occurrences)
No user-facing reset mechanism exists. In Slack specifically, slash commands like /new or /reset are intercepted by the Slack client as native commands (autocomplete shows /remind etc.) and never reach OpenClaw — so the workarounds discussed in #21725, #27058, #65200 are unavailable on this channel.
Manual procedure on the gateway host (used successfully both times):
# 1. Diagnose — confirm a pinned override exists
python3 -c "import jsonp = '/home/openclaw/.openclaw/agents/main/sessions/sessions.json'with open(p) as f: d = json.load(f)k = 'agent:main:main'if k in d: s = d[k] print(f\"sessionId: {s.get('sessionId')}\") print(f\"provider: {s.get('provider')}\") print(f\"model: {s.get('model')}\")else: print('agent:main:main not present')"# 2. Stop gateway
systemctl stop openclaw
# 3. Move transcript aside (reversible)
mv ~/.openclaw/agents/main/sessions/<SESSION_UUID>.jsonl \
~/.openclaw/agents/main/sessions/<SESSION_UUID>.jsonl.aside
# 4. Backup sessions.json
cp ~/.openclaw/agents/main/sessions/sessions.json \
~/.openclaw/agents/main/sessions/sessions.json.backup-$(date +%Y%m%d)# 5. Remove the pinned key (Python preserves valid JSON)
python3 -c "import jsonp = '/home/openclaw/.openclaw/agents/main/sessions/sessions.json'with open(p) as f: d = json.load(f)d.pop('agent:main:main', None)with open(p, 'w') as f: json.dump(d, f, indent=2)"# 6. Restart
systemctl start openclaw
Verification after restart
Send any short message in Slack DM (e.g. "Which LLM are you using?"). Expected result:
Response in seconds (not minutes).
Model self-identifies as the configured primary.
Both occurrences resolved within ~60 seconds of gateway start.
Detection signals for operators
If you suspect this is happening in your deployment:
Latency asymmetry across channels: one channel responds in seconds, another in minutes, with identical config. Consistent (not random) latency in the slow channel.
Log evidence: look for from=<non-primary> in embedded run failover decision log lines. from= is the starting model for the turn — if it's not your configured primary, the session is pinned.
Direct inspection: read agent:<id>:<lane> keys in sessions.json. Any model field that differs from agents.defaults.model.primary (or the agent-level override) indicates a pin.
Recommendations for upstream
In priority order:
Periodic primary probe for pinned sessions. When session state has a model differing from the configured primary, throttle-retry the primary (e.g. once per N minutes or once per N turns). Mirror the existing provider-cooldown probe behavior at the session-state level.
Treat session model as a soft preference. When the configured primary is healthy, try it first regardless of session state. Only fall through to the session-pinned model if the primary fails for the specific turn.
Stricter persistence criteria. Do not persist a fallback model to session state if the turn produced an empty / no-visible-reply response. The current behavior captured Grok with provider: null after Grok returned no visible content — this is dirty state that should be rejected on the write path.
Occurrence 2 surfaced a distinct concern: xai/grok-4-1-fast-non-reasoning returned no visible content on the turn that triggered re-pinning. The fallback engine apparently treated the API-level completion as success and persisted the model anyway. Two follow-up actions for maintainers:
Decide whether "empty response body" should be classified as a turn failure for the purposes of fallback selection and session-state persistence.
Consider whether provider: null should be allowed to be written at all, or rejected with a logged warning.
Unbounded session growth (observed in occurrence 1: 1.87 MB transcript)
This family of issues all stem from the same root: persisted session model state that does not reconcile with the configured primary, with no first-class mechanism for visibility, reset, or recovery. A unified fix on the persistence + reconciliation layer would address all of them.
Reproducibility
The workaround has worked reproducibly across two occurrences, two OpenClaw releases, two different pinned models, and one anomalous provider: null state. Operators encountering the same symptoms can apply the procedure with high confidence; the operations are reversible, the only file modified in place is sessions.json (which is backed up first), and per-channel sessions other than the affected one are untouched.
Happy to provide raw log excerpts or screenshots if useful for debugging upstream — the persisted sessions.json entries and the embedded run failover decision log lines from both occurrences are available.
Bug type
Behavior bug (incorrect output/state without crash). State-persistence regression.
Summary
When a model fallback occurs during a turn (primary temporarily unavailable due to rate limit, billing, or timeout), the fallback model is persisted in the session state stored in
sessions.json. Once persisted, that session uses the pinned model for every future turn indefinitely, with no mechanism to retry the configured primary even after the primary has fully recovered.This is the session-store counterpart to #47705 (same pathology at the
openclaw.jsonagent-config level). The underlying defect is the same: a model selection made under temporary failure conditions becomes permanent and silently overrides user configuration.Confirmed on two separate occurrences, three weeks apart, across two OpenClaw releases. Same single-user environment, same agent, same fallback chain. Different pinned model each time.
Environment
2026.4.15 (041266a)2026.5.6 (c97b9f7)22.22.2mainagent shared across all channelsanthropic/claude-haiku-4-5-20251001[google/gemini-3.1-flash-lite-preview, xai/grok-4-1-fast-non-reasoning]Occurrence 1 — 24 April 2026 (v2026.4.15)
Symptom
Asymmetric production behavior, identical config across channels:
The asymmetry persisted across gateway restarts and a full server reboot. Only the per-channel session state differed.
Root cause confirmed
The Slack DM session in
~/.openclaw/agents/main/sessions/sessions.jsonhad a persisted model entry:Every inbound Slack message read this state, skipped Haiku entirely, and started the turn on Gemini. Gateway log evidence:
The
from=google/gemini-...value confirms Gemini was the starting model for the turn, not a fallback from Haiku.WhatsApp sessions never accumulated the pinning because that channel's session key is separate and had been created under healthier conditions.
Occurrence 2 — 16 May 2026 (v2026.5.6)
Symptom
Slack DM error message visible to the user:
State observed
After three weeks of normal operation and no manual changes to
sessions.json, the Slack DM session had been re-pinned. This time the pinned state was different from occurrence 1:Two notable differences vs occurrence 1:
providerisnull— schema-anomalous (should bexai). The session entry was written with an incomplete provider attribution.Interpretation of the user-facing error
Reading the error message together with the persisted state suggests the following sequence:
providerfield unset.The persistence path treats "model that completed the turn at the API level" as success even when the response body is empty. This is an independent secondary issue worth flagging (see "Related secondary issue" below).
Files associated with the pinned session in v2026.5.6
Occurrence 2 surfaced additional per-session files that were not present (or not noted) in occurrence 1:
The workaround below moves only the transcript
.jsonlaside (as in April). The trajectory files were not touched and produced no observable post-restart issue. Maintainers may want to confirm whether trajectory files should also be archived as part of any future reset procedure.Why automatic recovery never happens
By the time either occurrence was diagnosed, the configured primary (Haiku) was fully operational — verified by:
But the Slack session never retried Haiku. There is no periodic reconciliation along the lines of:
The session holds the pinned model forever, until manually cleared at the file-system level.
The docs at
/concepts/model-failoverdescribe primary-probing behavior at the provider-cooldown level ("The primary candidate may be probed near cooldown expiry"). That mechanism does not apply to session-level pinned overrides.Workaround (reversible, production-safe, reproducible across both occurrences)
No user-facing reset mechanism exists. In Slack specifically, slash commands like
/newor/resetare intercepted by the Slack client as native commands (autocomplete shows/remindetc.) and never reach OpenClaw — so the workarounds discussed in #21725, #27058, #65200 are unavailable on this channel.Manual procedure on the gateway host (used successfully both times):
Verification after restart
Send any short message in Slack DM (e.g. "Which LLM are you using?"). Expected result:
Both occurrences resolved within ~60 seconds of gateway start.
Detection signals for operators
If you suspect this is happening in your deployment:
from=<non-primary>inembedded run failover decisionlog lines.from=is the starting model for the turn — if it's not your configured primary, the session is pinned.agent:<id>:<lane>keys insessions.json. Anymodelfield that differs fromagents.defaults.model.primary(or the agent-level override) indicates a pin.Recommendations for upstream
In priority order:
Periodic primary probe for pinned sessions. When session state has a model differing from the configured primary, throttle-retry the primary (e.g. once per N minutes or once per N turns). Mirror the existing provider-cooldown probe behavior at the session-state level.
Treat session model as a soft preference. When the configured primary is healthy, try it first regardless of session state. Only fall through to the session-pinned model if the primary fails for the specific turn.
Stricter persistence criteria. Do not persist a fallback model to session state if the turn produced an empty / no-visible-reply response. The current behavior captured Grok with
provider: nullafter Grok returned no visible content — this is dirty state that should be rejected on the write path.Surface overrides to users / operators. Expose pinned sessions via
openclaw statusor a dedicatedopenclaw sessions listcommand. Tracked partially in Feature: Session-level model overrides should be visible and clearable in Dashboard/CLI #22443.Channel-agnostic reset path. Provide a CLI command such as
openclaw sessions reset-model --key <sessionKey>for operators in channels that intercept slash commands (Slack, Nextcloud Talk per [Feature] Add CLI command to reset a single session #48016). Tracked partially in [Feature] Add CLI command to reset a single session #48016, Feature request: session-scoped reset / forget for chat and group contexts #50790.Related secondary issue worth flagging separately
Occurrence 2 surfaced a distinct concern:
xai/grok-4-1-fast-non-reasoningreturned no visible content on the turn that triggered re-pinning. The fallback engine apparently treated the API-level completion as success and persisted the model anyway. Two follow-up actions for maintainers:provider: nullshould be allowed to be written at all, or rejected with a logged warning.Related issues (same root cause family)
openclaw.jsonagent-config level/new//resetdo not clear overrides (narrower scope)/newinitializes fresh session on wrong defaultThis family of issues all stem from the same root: persisted session model state that does not reconcile with the configured primary, with no first-class mechanism for visibility, reset, or recovery. A unified fix on the persistence + reconciliation layer would address all of them.
Reproducibility
The workaround has worked reproducibly across two occurrences, two OpenClaw releases, two different pinned models, and one anomalous
provider: nullstate. Operators encountering the same symptoms can apply the procedure with high confidence; the operations are reversible, the only file modified in place issessions.json(which is backed up first), and per-channel sessions other than the affected one are untouched.Happy to provide raw log excerpts or screenshots if useful for debugging upstream — the persisted
sessions.jsonentries and theembedded run failover decisionlog lines from both occurrences are available.