Bug type

Behavior bug (incorrect output/state without crash)

Summary

When the primary model fails and a fallback model handles the request, the
fallback model gets persisted back to openclaw.json under agents.list[].model,
permanently replacing the configured primary. After this happens, the primary
model is never retried — even after it recovers — because the config itself has
been overwritten.

Steps to reproduce

1. Configure an agent with a primary model (e.g., openai-codex/gpt-5.3-codex)
   and fallbacks (e.g., openrouter/auto, xai/grok-4-1-fast-reasoning)
2. Wait for the primary model to become temporarily unavailable (rate limit,
   timeout, etc.)
3. A fallback model handles the request successfully
4. Check openclaw.json — agents.list[].model for the affected agent has been
   changed from the primary to the fallback
5. Restart the gateway — the agent continues using the fallback, never retrying
   the primary

Expected behavior

• Fallback models should be used temporarily when the primary is unavailable
  • The primary model in openclaw.json agents.list[].model should never be
    overwritten by a fallback
  • When the primary model recovers, it should be retried on subsequent requests

Actual behavior

• setSessionRuntimeModel() stores the fallback model in the session entry
  (entry.model, entry.modelProvider)
  • resolveSessionModelRef() prioritizes the session's stored runtime model over
    the configured primary, so the fallback becomes sticky
  • applyAgentConfig() writes the fallback model back to openclaw.json
agents.list[].model, making the change permanent
  • The primary model is never retried, even across gateway restarts

Root Cause (from source investigation)

The issue flows through three functions:

Suggested Fix

Any of the following would resolve this:

1. Don't persist fallback models as the session runtime model — only store the
   model that was configured, not the one that happened to succeed
2. Don't write session runtime models back to agents.list[].model in
   openclaw.json — session state should not overwrite agent config
3. Always prefer the configured primary model in resolveSessionModelRef()
   and only fall back when it actively fails on that request
4. Add a retry/reset mechanism that periodically retries the primary model
   instead of permanently sticking with the fallback

Workaround

Manually fix openclaw.json and clear the stale session:

1. Stop the gateway: systemctl --user stop openclaw-gateway.service
2. Edit openclaw.json — restore agents.list[].model to the correct primary
3. Delete the stale session file in
   ~/.openclaw/agents/<agent>/sessions/<uuid>.jsonl
4. Remove the session key mapping from sessions.json
5. Restart the gateway

OpenClaw version

OpenClaw version: 2026.3.13

Operating system

Ubuntu 25.10 (Linux 6.17.0-14-generic) -Zorin OS 18 Core

Install method

npm global

Model

openai-codex/gpt-5.3-codex

Provider / routing chain

Primary: openai-codex/gpt-5.3-codex Fallbacks (in order): 1. xai/grok-4-1-fast-reasoning 2. openrouter/auto 3. nvidia/moonshotai/kimi-k2.5 4. lmstudio/qwen/qwen3.5-35b-a3b 5. openai-codex/gpt-5.4

Config file / key location

Config file: ~/.openclaw/openclaw.json Key paths: Primary model: models.providers..models[].id Agent model: agents.list[].model Default primary: agents.defaults.model.primary Fallback chain: agents.defaults.model.fallbacks[] Session store: ~/.openclaw/agents//sessions/sessions.json Session files: ~/.openclaw/agents//sessions/.jsonl

Additional provider/model setup details

Additional Context

• models.mode is set to "merge" (custom providers merge with built-ins)
• agents.defaults.contextTokens: 131072
• agents.defaults.maxConcurrent: 4
• Gateway mode: local (loopback, port 18789)
• Channels: telegram (active), whatsapp (active but experiencing WebSocket
  failures)
• Cron jobs run on the qwen agent (local LMStudio) — not affected by this bug
• The main agent is the only one observed to have its config overwritten

Logs, screenshots, and evidence

the config-audit.jsonl shows every config write. You can see you've
  had to manually reset the primary model multiple times. Add this to the bug
  report:

  ## Logs

  ### config-audit.jsonl (redacted)

  The config audit log shows repeated manual corrections to restore the
  primary model, indicating the bug has occurred multiple times:

  2026-03-11T19:59:51Z config.write — "config set agents.defaults.model.primary
  openai-codex/gpt-5.3-codex"
  2026-03-11T20:02:39Z config.write — "config set agents.defaults.model.primary
  openai-codex/gpt-5.3-codex-thinking"
  2026-03-13T02:13:32Z config.write — "config set agents.defaults.model.primary
  openai-codex/gpt-5.3-codex-thinking"
  2026-03-13T02:18:48Z config.write — "config set agents.defaults.model.primary
  openai-codex/gpt-5.3-codex"
  2026-03-13T04:49:51Z config.write — "config set agents.defaults.model.primary
  openai-codex/gpt-5.3-codex"
  2026-03-13T05:06:18Z config.write — "config set agents.defaults.model.primary
  openai-codex/gpt-5.4-codex"
  2026-03-13T05:09:30Z config.write — "config set agents.defaults.model.primary
  openai-codex/gpt-5.3-codex"

  Note: No corresponding `config.write` entries from the gateway process
  itself changing `agents.list[].model` — the overwrite to `agents.list`
  may happen via in-memory state persisted on shutdown rather than through
  the config CLI, making it harder to trace.

  ### journalctl (gateway)

  2026-03-15T18:51:51Z [reload] config change detected; evaluating reload
    (models.providers.xai.models, agents.defaults.model.fallbacks,
     agents.defaults.models.xai/grok-4-1-fast-reasoning, agents.list)
  2026-03-15T18:51:51Z [reload] config hot reload applied

  The gateway's hot-reload confirms it watches `agents.list` for changes,
  but does not log when it *writes* model changes back to that key.

Impact and severity

Impact

Affected

• All agents using a fallback chain (primarily the main agent)
• All channels (Telegram, WhatsApp) — messages route through the wrong model
• All users interacting with the affected agent

Severity

Blocks workflow. The configured primary model is silently replaced.
The user gets no notification that the model has changed. The only way
to discover it is to manually inspect openclaw.json or notice
degraded response quality. Requires manual config editing and session
clearing to fix — non-technical users would be stuck.

Frequency

Intermittent but recurring. Triggers whenever the primary model has
a transient failure (rate limit, timeout, maintenance window). In our
case it has happened at least 6 times over 5 days, requiring repeated
manual corrections (see config-audit.jsonl).

Consequences

• Wrong model serving all requests — responses differ in quality,
  capability, and behavior without the user's knowledge
• Extra cost — fallback to a paid provider (e.g., OpenRouter) when
  the primary (included with subscription) is available again
• Lost primary model access — the primary is never retried, even
  after it fully recovers
• Cron job interference — if the main agent's model changes, any
  cron jobs or automations relying on that agent get the wrong model
• Repeated manual intervention — the only fix is editing config
  files and clearing sessions by hand, every time it happens

Additional information

• This is not a confirmed regression — the behavior has been present
  since at least v2026.3.11 (first version installed). Cannot confirm
  whether earlier versions are affected. Current version is v2026.3.13.

  • The bug is easier to trigger when the primary model provider has
    intermittent availability (e.g., OpenAI Codex during peak hours or
    maintenance windows). A single transient failure is enough to
    permanently lock the agent to the fallback.

  • The agents.defaults.model.primary key is NOT overwritten — only
    agents.list[].model on the specific agent. This means the default
    config looks correct, but the per-agent override silently takes
    precedence, making the bug harder to spot.

  • The config-audit.jsonl does not capture the gateway's internal write
    that overwrites agents.list[].model. All logged writes are from
    manual openclaw config set commands. This suggests the gateway
    writes the config through a code path that either bypasses the audit
    logger or batches the write with other state on shutdown/reload.

  • Installed via npm (~/.npm-global/), running as a systemd user
    service on Ubuntu 25.10. Node v25.6.1 (Linuxbrew).

  • The qwen agent (local LMStudio, no fallback chain) has never been
    affected — consistent with the bug requiring a fallback to trigger.