Skip to content

fix(models): unify provider runtime selection#71176

Closed
vincentkoc wants to merge 4 commits into
mainfrom
fix/model-harness-separation
Closed

fix(models): unify provider runtime selection#71176
vincentkoc wants to merge 4 commits into
mainfrom
fix/model-harness-separation

Conversation

@vincentkoc

@vincentkoc vincentkoc commented Apr 24, 2026

Copy link
Copy Markdown
Member

Summary

  • migrate legacy runtime-backed model refs (codex/*, codex-cli/*, claude-cli/*, google-gemini-cli/*) back to canonical provider/model refs while preserving the selected runtime separately
  • route canonical OpenAI, Anthropic, and Google model refs through the selected embedded harness or CLI backend instead of relying on fake providers
  • expose compatible runtime choices through the shared model provider data and Discord model picker; update Mattermost slash hints for --runtime
  • update Anthropic Claude CLI and Google Gemini CLI setup/migration to seed canonical model refs plus runtime config
  • keep the PR rebased on feat(plugins): report setup descriptor drift #71194 setup descriptor drift changes

Validation

  • OPENCLAW_LOCAL_CHECK=0 pnpm check:changed
  • pnpm plugin-sdk:api:check

@vincentkoc vincentkoc self-assigned this Apr 24, 2026
@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation gateway Gateway runtime commands Command implementations agents Agent runtime and tooling size: M maintainer Maintainer-authored PR labels Apr 24, 2026
@vincentkoc
vincentkoc marked this pull request as ready for review April 24, 2026 17:11
@aisle-research-bot

aisle-research-bot Bot commented Apr 24, 2026

Copy link
Copy Markdown

🔒 Aisle Security Analysis

We found 3 potential security issue(s) in this PR:

# Severity Title
1 🟠 High Legacy codex/* model migration can silently route prompts to non-Codex harness (e.g., embedded PI/OpenAI)
2 🟡 Medium Prototype pollution via user-controlled keys in legacy agent models allowlist migration
3 🟡 Medium Terminal/log injection (CWE-117) via unsanitized agent IDs in doctor change notes
1. 🟠 Legacy codex/* model migration can silently route prompts to non-Codex harness (e.g., embedded PI/OpenAI)
Property Value
Severity High
CWE CWE-200
Location src/commands/doctor/shared/legacy-config-core-normalizers.ts:227-242

Description

normalizeLegacyCodexHarnessModelRefs() migrates codex/* model references to openai/*. It attempts to ensure the Codex harness is selected, but only when embeddedHarness.runtime is missing or "auto".

As a result:

  • If a user previously had agents.*.embeddedHarness.runtime explicitly set to a non-auto value (e.g. "pi" or some other plugin harness id) and had a codex/* primary model selected, the migration will rewrite the model ref to openai/* but will not update the harness runtime.
  • After migration, execution may run through the forced non-Codex harness (not the Codex app-server harness), potentially changing authentication route and data handling (e.g., sending prompts/media to the OpenAI API via embedded PI rather than the bounded Codex harness).
  • This migration runs during config read via applyRuntimeLegacyConfigMigrations() (used in src/config/io.ts), so the behavior change can occur automatically on startup.

Vulnerable code:

function ensureCodexEmbeddedHarness(raw: AgentEmbeddedHarnessPatch | undefined) {
  ...
  const runtime = normalizeOptionalLowercaseString(raw.runtime);
  if (!runtime || runtime === "auto") {
    return { value: { ...raw, runtime: "codex" }, ... };
  }
  return { value: raw, changed: false }; // non-auto runtime blocks forcing codex
}

Recommendation

When migrating a codex/* primary model to openai/*, ensure the run remains bound to the Codex harness even if an explicit non-auto embeddedHarness.runtime was set, or at minimum require explicit user confirmation / emit a high-visibility warning.

Safer options:

  1. Force Codex harness on codex-selected primary, overriding any existing runtime that is not "codex" (or store the prior runtime elsewhere):
function ensureCodexEmbeddedHarness(raw: AgentEmbeddedHarnessPatch | undefined) {
  if (!isRecord(raw)) return { value: { runtime: "codex" }, changed: true };
  const runtime = normalizeOptionalLowercaseString(raw.runtime);
  if (runtime !== "codex") {
    return { value: { ...raw, runtime: "codex" }, changed: true };
  }
  return { value: raw, changed: false };
}
  1. If overriding is not acceptable, do not migrate codex/* → openai/* when runtime is explicitly non-Codex; instead:
  • keep the legacy ref, or
  • migrate but add a blocking validation error / interactive prompt, or
  • add a changes.push(...) warning stating that the harness runtime prevented selecting Codex and that traffic may go to OpenAI.

Also consider adding a regression test for embeddedHarness.runtime: "pi" + model: "codex/..." to ensure the chosen behavior is explicit and safe.

2. 🟡 Prototype pollution via user-controlled keys in legacy agent models allowlist migration
Property Value
Severity Medium
CWE CWE-1321
Location src/commands/doctor/shared/legacy-config-core-normalizers.ts:305-317

Description

normalizeLegacyCodexAllowlistModels() builds a plain object ({}) and then assigns properties using keys taken directly from a user-controlled config object (rawModels).

Because next is a normal object with Object.prototype, assigning attacker-chosen keys like __proto__, constructor, or prototype can mutate the prototype of next (or create surprising inherited properties), i.e. prototype pollution.

  • Input: rawModels from cfg.agents.defaults.models / cfg.agents.list[*].models (loaded from config)
  • Transformation: Object.entries(rawModels) iterates keys without filtering
  • Sink: next[rawKey] = ... / next[migratedKey] = ... on {}

Vulnerable code:

const next: AgentModelsMapPatch = {};
for (const [rawKey, entry] of Object.entries(rawModels)) {
  ...
  next[rawKey] = mergeModelEntry(entry, next[rawKey]);
}

If this polluted object is later merged/spread or used in security-sensitive decisions (e.g., allowlisting models), it can cause incorrect behavior or potentially impact other parts of the process relying on object properties.

Recommendation

Harden the migration map against special keys.

Option A (recommended): use a null-prototype object and reject dangerous keys:

const next: Record<string, AgentModelsMapPatch[string]> = Object.create(null);
const isSafeKey = (k: string) => !["__proto__","prototype","constructor"].includes(k);

for (const [rawKey, entry] of Object.entries(rawModels)) {
  if (!isSafeKey(rawKey)) continue;
  const migratedKey = migrateLegacyCodexModelRef(rawKey);
  const key = migratedKey ?? rawKey;
  if (!isSafeKey(key)) continue;
  next[key] = mergeModelEntry(entry, next[key]);
}

Option B: use Map<string, ...> during normalization and convert to an object only at the boundary (also with Object.create(null)), ensuring no prototype setters are invoked.

Also consider validating that model keys match an expected pattern like /^[a-z0-9-]+\/[\w.-]+$/i before inserting them.

3. 🟡 Terminal/log injection (CWE-117) via unsanitized agent IDs in doctor change notes
Property Value
Severity Medium
CWE CWE-117
Location src/commands/doctor-config-flow.ts:115-123

Description

openclaw doctor emits "changes" messages that interpolate config-controlled values (agent IDs) into terminal output without sanitization.

  • Input (untrusted): entry.id comes from the user-controlled config (cfg.agents.list[].id).
  • Propagation: normalizeLegacyCodexHarnessModelRefs() builds path using agentId and passes it into normalizeLegacyCodexAgentContainer(), which pushes human-readable change messages containing that path.
  • Sink: loadAndMaybeMigrateDoctorConfig() prints normalized.changes via note(normalized.changes.join("\n"), "Doctor changes") without calling sanitizeDoctorNote() / sanitizeForLog().

If entry.id contains control characters (e.g., \n, \r) or ANSI escape sequences, it can:

  • forge additional lines in the output (log/terminal forging)
  • inject terminal control codes (hide warnings, spoof success, manipulate cursor)

Vulnerable code paths:

const agentId = normalizeOptionalString(entry.id) ?? String(index);
const agent = normalizeLegacyCodexAgentContainer(entry, `agents.list.${agentId}`, changes);// ...
changes.push(`Moved ${path}.model codex/* refs → openai/* ...`);

printed later as:

note(normalized.changes.join("\n"), "Doctor changes");

Recommendation

Sanitize any user/config-derived content before emitting it to terminal/log output.

Option A (preferred): sanitize the entire note payload at the sink

import { sanitizeDoctorNote } from "./doctor/emit-notes.js";// ...
note(sanitizeDoctorNote(normalized.changes.join("\n")), "Doctor changes");

Option B: sanitize path components where they are constructed

import { sanitizeForLog } from "../../../terminal/ansi.js";

const agentIdRaw = normalizeOptionalString(entry.id) ?? String(index);
const agentId = sanitizeForLog(agentIdRaw);

Also consider validating agents.list[].id against a safe identifier regex (e.g., /^[a-z0-9_-]+$/i) and rejecting or escaping anything else.


Analyzed PR: #71176 at commit 840cb42

Last updated on: 2026-04-24T18:05:43Z

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 713a3e48b7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +277 to +278
changed = true;
codexSelected = true;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not force Codex harness for migrated fallback refs

Setting codexSelected = true for migrated fallback entries causes normalizeLegacyCodexAgentContainer to pin embeddedHarness.runtime to "codex" for the whole agent, even when the primary model was not a Codex alias. In a config like primary: "openai/..." with a legacy codex/... fallback, migration now changes all turns to forced Codex runtime instead of keeping the previous behavior where only the Codex fallback used that harness; this can change auth/runtime semantics and break primary models that are not intended for the Codex harness.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in current head b8ebe38e28: fallback-only legacy runtime refs are migrated to canonical provider refs but do not select embeddedHarness.runtime. Only legacy primary/string model refs select the runtime. Covered by does not force Codex harness when only fallback refs are migrated in src/commands/doctor-legacy-config.migrations.test.ts.

@greptile-apps

greptile-apps Bot commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR cleanly separates the virtual codex provider from real model-picker surfaces and adds an automatic config migration that rewrites legacy codex/* model refs to openai/* while preserving Codex harness behavior via embeddedHarness.runtime: "codex". The implementation is well-structured and the new test coverage is thorough, but there is one order-dependent merge bug in the allowlist migration that can silently drop canonical openai/* config values.

  • P1 merge priority bug: normalizeLegacyCodexAllowlistModels uses iteration order to decide which of a codex/* / openai/* pair wins for conflicting keys — the canonical openai/* entry loses whenever it appears after the codex/* key in the config JSON.

Confidence Score: 4/5

Safe to merge after resolving the order-dependent merge priority bug in the allowlist normalizer, which can silently drop canonical openai/* config values.

One P1 correctness defect exists in normalizeLegacyCodexAllowlistModels where the merge outcome depends on JSON key order rather than semantic priority. All other findings are P2 or cosmetic. The picker visibility logic, harness migration, and test coverage are sound.

src/commands/doctor/shared/legacy-config-core-normalizers.ts (mergeModelEntry call site in normalizeLegacyCodexAllowlistModels)

Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/commands/doctor/shared/legacy-config-core-normalizers.ts
Line: 307-315

Comment:
**Order-dependent merge silently drops canonical `openai/*` values**

`mergeModelEntry(entry, next[migratedKey])` places the incoming entry as `legacyEntry` and whatever is already in `next` as `currentEntry`. Because the spread is `{ ...legacyEntry, ...currentEntry }`, the `currentEntry` wins — but what is in `next` at that moment depends entirely on JSON key order.

When `codex/gpt-5.5` appears before `openai/gpt-5.5` in the user's config, the migration processes the codex key first (places its values into `next`), then processes the openai key as the "legacy" entry — so the codex alias/params overwrite the canonical openai ones. Reversing the order gives the correct result. Users whose JSON has `codex/*` keys ordered before `openai/*` keys will silently lose their canonical `openai/*` overrides after migration.

The fix is to always let the canonical `openai/*` entry win: collect non-codex entries first, then merge migrated codex entries as the lower-priority source.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/commands/doctor/shared/legacy-config-core-normalizers.ts
Line: 349-358

Comment:
**Misleading change message when harness is modified then cleaned up in the same pass**

When a codex model is selected and the existing harness is `{ runtime: "auto", fallback: "pi" }`, `ensureCodexEmbeddedHarness` first rewrites it to `{ runtime: "codex", fallback: "pi" }`. Then `normalizeNoopEmbeddedHarness` strips `fallback: "pi"` (a default) and emits `"Removed no-op agents.defaults.embeddedHarness defaults."` The migration log therefore says both "selected Codex harness" and "removed no-op defaults" for the same field in the same run, which can make the audit trail confusing — the `fallback` removal isn't a separate concern but a side-effect of the harness selection step. Consider suppressing the no-op message (or merging it into the harness selection message) when `model.codexSelected` is true.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "Merge branch 'main' into fix/model-harne..." | Re-trigger Greptile

Comment on lines +307 to +315
for (const [rawKey, entry] of Object.entries(rawModels)) {
const migratedKey = migrateLegacyCodexModelRef(rawKey);
if (migratedKey) {
changed = true;
next[migratedKey] = mergeModelEntry(entry, next[migratedKey]);
continue;
}
next[rawKey] = mergeModelEntry(entry, next[rawKey]);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Order-dependent merge silently drops canonical openai/* values

mergeModelEntry(entry, next[migratedKey]) places the incoming entry as legacyEntry and whatever is already in next as currentEntry. Because the spread is { ...legacyEntry, ...currentEntry }, the currentEntry wins — but what is in next at that moment depends entirely on JSON key order.

When codex/gpt-5.5 appears before openai/gpt-5.5 in the user's config, the migration processes the codex key first (places its values into next), then processes the openai key as the "legacy" entry — so the codex alias/params overwrite the canonical openai ones. Reversing the order gives the correct result. Users whose JSON has codex/* keys ordered before openai/* keys will silently lose their canonical openai/* overrides after migration.

The fix is to always let the canonical openai/* entry win: collect non-codex entries first, then merge migrated codex entries as the lower-priority source.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/commands/doctor/shared/legacy-config-core-normalizers.ts
Line: 307-315

Comment:
**Order-dependent merge silently drops canonical `openai/*` values**

`mergeModelEntry(entry, next[migratedKey])` places the incoming entry as `legacyEntry` and whatever is already in `next` as `currentEntry`. Because the spread is `{ ...legacyEntry, ...currentEntry }`, the `currentEntry` wins — but what is in `next` at that moment depends entirely on JSON key order.

When `codex/gpt-5.5` appears before `openai/gpt-5.5` in the user's config, the migration processes the codex key first (places its values into `next`), then processes the openai key as the "legacy" entry — so the codex alias/params overwrite the canonical openai ones. Reversing the order gives the correct result. Users whose JSON has `codex/*` keys ordered before `openai/*` keys will silently lose their canonical `openai/*` overrides after migration.

The fix is to always let the canonical `openai/*` entry win: collect non-codex entries first, then merge migrated codex entries as the lower-priority source.

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in current head b8ebe38e28: model allowlist migration is two-pass now, so canonical provider keys are collected first and migrated legacy keys merge as lower-priority entries. Covered by keeps canonical OpenAI model entries ahead of migrated Codex aliases in src/commands/doctor-legacy-config.migrations.test.ts.

Comment on lines +349 to +358
const harness = normalizeNoopEmbeddedHarness(next.embeddedHarness as AgentEmbeddedHarnessPatch);
if (harness.changed) {
if (harness.value) {
next.embeddedHarness = harness.value;
} else {
delete next.embeddedHarness;
}
changed = true;
changes.push(`Removed no-op ${path}.embeddedHarness defaults.`);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Misleading change message when harness is modified then cleaned up in the same pass

When a codex model is selected and the existing harness is { runtime: "auto", fallback: "pi" }, ensureCodexEmbeddedHarness first rewrites it to { runtime: "codex", fallback: "pi" }. Then normalizeNoopEmbeddedHarness strips fallback: "pi" (a default) and emits "Removed no-op agents.defaults.embeddedHarness defaults." The migration log therefore says both "selected Codex harness" and "removed no-op defaults" for the same field in the same run, which can make the audit trail confusing — the fallback removal isn't a separate concern but a side-effect of the harness selection step. Consider suppressing the no-op message (or merging it into the harness selection message) when model.codexSelected is true.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/commands/doctor/shared/legacy-config-core-normalizers.ts
Line: 349-358

Comment:
**Misleading change message when harness is modified then cleaned up in the same pass**

When a codex model is selected and the existing harness is `{ runtime: "auto", fallback: "pi" }`, `ensureCodexEmbeddedHarness` first rewrites it to `{ runtime: "codex", fallback: "pi" }`. Then `normalizeNoopEmbeddedHarness` strips `fallback: "pi"` (a default) and emits `"Removed no-op agents.defaults.embeddedHarness defaults."` The migration log therefore says both "selected Codex harness" and "removed no-op defaults" for the same field in the same run, which can make the audit trail confusing — the `fallback` removal isn't a separate concern but a side-effect of the harness selection step. Consider suppressing the no-op message (or merging it into the harness selection message) when `model.codexSelected` is true.

How can I resolve this? If you propose a fix, please make it concise.

@steipete

Copy link
Copy Markdown
Contributor

Thanks @vincentkoc. Codex review found the direction right, but this exact PR is unsafe to land and the branch is not maintainer-editable.

Main blocker: the migration treats any migrated codex/* fallback as a reason to set embeddedHarness.runtime = "codex" for the whole agent/defaults container. That changes configs like primary: "openai/..." with a legacy codex/... fallback into forced Codex-harness runs for the primary too. The model-map merge also depended on JSON key order, so canonical openai/* entries could lose to migrated legacy entries.

I rewrote this as #71193 with the narrower behavior:

  • hide virtual codex from model picker and /models provider surfaces
  • migrate legacy primary codex/* refs to openai/* plus explicit Codex harness policy
  • preserve fallback-only legacy codex/* refs instead of pinning the whole container
  • add regression tests for picker visibility, /models, primary migration, canonical merge precedence, and fallback-only safety

Closing this in favor of #71193.

@steipete steipete closed this Apr 24, 2026
@vincentkoc vincentkoc changed the title fix(models): split Codex harness from OpenAI model choices fix(models): unify provider runtime selection Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling commands Command implementations docs Improvements or additions to documentation gateway Gateway runtime maintainer Maintainer-authored PR size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants