Skip to content

fix(models): normalize provider runtime selection#71259

Merged
vincentkoc merged 7 commits into
mainfrom
fix/provider-runtime-selection-followup
Apr 24, 2026
Merged

fix(models): normalize provider runtime selection#71259
vincentkoc merged 7 commits into
mainfrom
fix/provider-runtime-selection-followup

Conversation

@vincentkoc

@vincentkoc vincentkoc commented Apr 24, 2026

Copy link
Copy Markdown
Member

Summary

  • reverses the codex-only model/harness migration from fix(models): separate Codex harness from model choices #71193
  • normalizes legacy runtime-backed provider refs (codex/*, codex-cli/*, claude-cli/*, google-gemini-cli/*) into canonical provider/model refs plus explicit runtime selection
  • keeps fallback-only legacy runtime refs unchanged because runtime is configured at the agent container level
  • defaults omitted embedded harness runtime to OpenClaw Pi; runtime: "auto" is now explicit opt-in for plugin harness auto-selection
  • updates channel/model selectors to show OpenClaw Pi Default as the PI/default runtime option
  • updates /status to report sandbox path as Execution: and model runtime as Runtime: OpenClaw Pi Default, Runtime: OpenAI Codex, or the selected CLI/ACP backend
  • aligns the tool-result middleware API from fix(plugins): harden tool result middleware #71241 around runtime terminology (runtime/runtimes, canonical codex id) while accepting the old codex-app-server harness id as a compatibility alias
  • hides runtime-backed legacy providers from model picker and /models provider surfaces, while exposing runtime choices separately for Discord/channel model selection
  • updates Anthropic/Gemini CLI setup and docs to use canonical model refs with runtime selection

Validation

  • OPENCLAW_LOCAL_CHECK=0 pnpm check:changed passed before the final rebase; after rebasing onto current origin/main, the rerun passed typecheck/lint/import/guard lanes and reached extension tests, then hung in vitest.extension-line with no CPU and was stopped
  • pnpm plugin-sdk:api:gen
  • pnpm plugin-sdk:api:check
  • pnpm config:schema:check
  • pnpm check:no-conflict-markers
  • tokenjuice wrap --raw -- pnpm check:docs
  • env OPENCLAW_LOCAL_CHECK=0 pnpm test src/agents/harness/tool-result-middleware.test.ts src/agents/codex-app-server.extensions.test.ts src/agents/pi-embedded-runner.extensions.test.ts extensions/codex/src/app-server/dynamic-tools.test.ts extensions/tokenjuice/index.test.ts extensions/tokenjuice/manifest.test.ts src/plugins/captured-registration.test.ts src/plugins/registry.test.ts
  • env OPENCLAW_LOCAL_CHECK=0 pnpm test src/auto-reply/reply/commands-status.test.ts
  • env OPENCLAW_LOCAL_CHECK=0 pnpm test src/auto-reply/status.test.ts src/status/status-message.test.ts
  • pnpm test src/agents/harness/selection.test.ts src/commands/doctor-legacy-config.migrations.test.ts src/auto-reply/reply/commands-models.test.ts extensions/discord/src/monitor/model-picker.test.ts extensions/discord/src/monitor/native-command.model-picker.test.ts src/config/schema.base.generated.test.ts src/config/schema.help.quality.test.ts src/agents/command/attempt-execution.cli.test.ts

@vincentkoc vincentkoc self-assigned this Apr 24, 2026
@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation channel: discord Channel integration: discord channel: mattermost Channel integration: mattermost gateway Gateway runtime commands Command implementations agents Agent runtime and tooling extensions: anthropic size: XL maintainer Maintainer-authored PR labels Apr 24, 2026
@vincentkoc
vincentkoc marked this pull request as ready for review April 24, 2026 21:00
@aisle-research-bot

aisle-research-bot Bot commented Apr 24, 2026

Copy link
Copy Markdown

🔒 Aisle Security Analysis

We found 2 potential security issue(s) in this PR:

# Severity Title
1 🟠 High User-controlled runtime override can force CLI backend execution (host credentials) for canonical providers
2 🟡 Medium Runtime override can pin unregistered harness and disable fallback, causing session-level DoS
1. 🟠 User-controlled runtime override can force CLI backend execution (host credentials) for canonical providers
Property Value
Severity High
CWE CWE-285
Location src/auto-reply/reply/agent-runner-execution.ts:936-987

Description

runAgentTurnWithFallback now remaps a canonical provider (e.g., anthropic) to a CLI backend provider (e.g., claude-cli) based on agentRuntimeOverride persisted in the session.

  • agentRuntimeOverride is sourced from the active session entry (which is updated from inline directives/model picker state).
  • resolveCliRuntimeExecutionProvider(...) can map provider=anthropic + runtimeOverride=claude-cli to cliExecutionProvider="claude-cli".
  • When isCliProvider(cliExecutionProvider, runtimeConfig) is true, execution is routed to runCliAgent(...), which will use the server host’s installed CLI backend and its local login/session.

Impact:

  • Any user who can influence agentRuntimeOverride for a session can potentially trigger local CLI execution (and thus reuse of host-level CLI auth) even when administrators expect requests to go through remote API-key based providers only.
  • This creates a security boundary break between “API provider access” and “host CLI backend access”, and can lead to unintended credential usage and policy bypass.

Vulnerable code (routing decision):

const agentRuntimeOverride = normalizeOptionalString(
  params.getActiveSessionEntry()?.agentRuntimeOverride,
);
const cliExecutionProvider =
  resolveCliRuntimeExecutionProvider({
    provider,
    cfg: runtimeConfig,
    agentId: params.followupRun.run.agentId,
    runtimeOverride: agentRuntimeOverride,
  }) ?? provider;

if (isCliProvider(cliExecutionProvider, runtimeConfig)) {
  ...
  const result = await runCliAgent({ provider: cliExecutionProvider, ... });
}

Recommendation

Treat switching to CLI backends as a privileged operation.

Suggested defenses (combine as appropriate):

  1. Authorization gate before allowing CLI routing
  • Only allow CLI execution when the sender is an owner/admin or the session is explicitly marked as trusted.
const wantsCli = isCliProvider(cliExecutionProvider, runtimeConfig);
if (wantsCli && !params.followupRun.run.senderIsOwner) {
  throw new Error("CLI backends are restricted to owners/admins");
}
  1. Config allowlist for per-workspace/per-channel runtime overrides
  • Require an explicit config flag like agents.defaults.allowUserRuntimeOverride: false (default false).
  1. Separate auth domains
  • Do not reuse host CLI sessions for untrusted users. Require per-user auth binding, or disable CLI backends entirely in multi-tenant deployments.
  1. Audit logging
  • Log whenever runtime override causes a provider to be executed via a CLI backend (include sessionKey, user identity, requested runtime).
2. 🟡 Runtime override can pin unregistered harness and disable fallback, causing session-level DoS
Property Value
Severity Medium
CWE CWE-400
Location src/agents/harness/selection.ts:282-293

Description

The session-scoped agentRuntimeOverride (set via the --runtime option / Discord model picker) is passed directly as agentHarnessId for embedded runs. The harness-selection logic treats any non-auto value as pinned and forces fallback: "none". If the selected runtime does not correspond to a registered harness (e.g., codex-cli, claude-cli when not installed), selection throws and the run fails, potentially leaving the session stuck in a broken state until the override is cleared.

Key points:

  • Input: --runtime <value> / Discord component r= → persisted into sessionEntry.agentRuntimeOverride
  • Propagation: agentRuntimeOverride is forwarded into embedded execution as agentHarnessId
  • Sink: selectAgentHarnessDecision throws when the pinned runtime is not registered and fallback is disabled

Vulnerable code (pinned runtime used as harness id):

...(agentRuntimeOverride && agentRuntimeOverride !== "auto" && agentRuntimeOverride !== "default"
  ? { agentHarnessId: agentRuntimeOverride }
  : {}),

Pinned-policy behavior that disables fallback:

if (!agentHarnessId?.trim()) return undefined;
const runtime = normalizeEmbeddedAgentRuntime(agentHarnessId);
if (runtime === "auto") return undefined;
return { runtime, fallback: "none" };

Throw on unregistered pinned harness:

if (policy.fallback === "none") {
  throw new Error(
    `Requested agent harness "${runtime}" is not registered and PI fallback is disabled.`,
  );
}

Recommendation

Do not treat user/session runtime overrides as a strict pin with fallback: "none" unless the runtime has been validated as available.

Options:

  1. Validate at persistence time: only persist agentRuntimeOverride if it resolves to an installed CLI backend or a registered harness; otherwise ignore and notify the user.
  2. Make pinned overrides safe by default: keep fallback enabled for session overrides.

Example (safe pinned policy):

function resolvePinnedAgentHarnessPolicy(agentHarnessId?: string): AgentHarnessPolicy | undefined {
  const raw = agentHarnessId?.trim();
  if (!raw) return undefined;
  const runtime = normalizeEmbeddedAgentRuntime(raw);
  if (runtime === "auto") return undefined;// Allow PI fallback for session overrides to avoid bricking sessions when a runtime is unavailable
  return { runtime, fallback: "pi" };
}

Additionally, consider generating runtimeChoicesByProvider based on available runtimes (registered harnesses / configured CLI backends) rather than static legacy aliases.


Analyzed PR: #71259 at commit f5c687e

Last updated on: 2026-04-24T23:57:36Z

@greptile-apps

greptile-apps Bot commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR reverses the codex-only PR #71193 and generalises the model/runtime split to Anthropic Claude CLI and Google Gemini CLI as well: canonical provider/model refs are kept for model selection while execution backend is recorded separately in embeddedHarness.runtime. It introduces model-runtime-aliases.ts as the central registry, extends the Discord model picker with a runtime selector row, and propagates --runtime through inline directives and session state.

Confidence Score: 5/5

Safe to merge; both remaining findings are minor defensive/style issues that don't affect current runtime behaviour.

The core migration logic, CLI routing, session-override persistence, and Discord picker changes are all well-tested. The only findings are a missing .startsWith guard (currently harmless given how the constant is defined) and a duplicate config label.

extensions/anthropic/register.runtime.ts (defensive coding), src/config/schema.labels.ts (duplicate label)

Prompt To Fix All With AI
This is a comment left during a code review.
Path: extensions/anthropic/register.runtime.ts
Line: 81-83

Comment:
`CLAUDE_CLI_CANONICAL_DEFAULT_MODEL_REF` slices `CLAUDE_CLI_DEFAULT_MODEL_REF` without first verifying it starts with `${CLAUDE_CLI_BACKEND_ID}/`, whereas the parallel `CLAUDE_CLI_CANONICAL_ALLOWLIST_REFS` computation directly above it correctly guards with `.startsWith(...)`. If `CLAUDE_CLI_DEFAULT_MODEL_REF` were ever changed to a canonical `anthropic/...` form, the slice would produce `anthropic/ic/claude-opus-4-7` or similar garbage silently.

```suggestion
const CLAUDE_CLI_CANONICAL_DEFAULT_MODEL_REF = CLAUDE_CLI_DEFAULT_MODEL_REF.startsWith(
  `${CLAUDE_CLI_BACKEND_ID}/`,
)
  ? `anthropic/${CLAUDE_CLI_DEFAULT_MODEL_REF.slice(CLAUDE_CLI_BACKEND_ID.length + 1)}`
  : CLAUDE_CLI_DEFAULT_MODEL_REF;
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/config/schema.labels.ts
Line: 75-76

Comment:
Both `agents.defaults.embeddedHarness` (the container object) and `agents.defaults.embeddedHarness.runtime` (the leaf field) are now labeled identically as `"Default Agent Runtime"`. In any flat config UI or help output that lists both entries, the two are indistinguishable. The old names ("Default Embedded Harness" / "Default Embedded Harness Runtime") were distinct; something like `"Default Agent Harness"` for the parent would avoid the collision.

```suggestion
  "agents.defaults.embeddedHarness": "Default Agent Harness",
  "agents.defaults.embeddedHarness.runtime": "Default Agent Runtime",
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "fix(models): reverse codex-only runtime ..." | Re-trigger Greptile

Comment thread extensions/anthropic/register.runtime.ts Outdated
Comment thread src/config/schema.labels.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b1507f267c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/auto-reply/reply/directive-handling.persist.ts Outdated
@vincentkoc
vincentkoc force-pushed the fix/provider-runtime-selection-followup branch 2 times, most recently from 996b280 to ef64be3 Compare April 24, 2026 22:31
@openclaw-barnacle openclaw-barnacle Bot added extensions: codex extensions: tokenjuice Changes to the bundled tokenjuice extension labels Apr 24, 2026
@vincentkoc
vincentkoc force-pushed the fix/provider-runtime-selection-followup branch 2 times, most recently from 1545408 to 72e6a72 Compare April 24, 2026 22:35
@steipete

Copy link
Copy Markdown
Contributor

Codex review: do not merge yet.

I agree with the direction of splitting canonical provider/model selection from runtime selection, but the current PR still has a real trust-boundary bug around /model --runtime.

Blocking finding:

  • src/auto-reply/reply/directive-handling.persist.ts:205 persists directives.rawModelRuntime from chat text directly into sessionEntry.agentRuntimeOverride for any parsed /model ... --runtime ... directive.
  • src/auto-reply/reply/agent-runner-execution.ts:936 then reads that session value and uses it to affect runtime selection. It can select a CLI execution provider through resolveCliRuntimeExecutionProvider(...), and for embedded runs it passes the same value through as agentHarnessId at src/auto-reply/reply/agent-runner-execution.ts:1106.
  • src/auto-reply/model.ts:18 accepts broad runtime strings ([A-Za-z0-9_.:-]+). I do not see an allowlist/permission check at persistence time tying the requested runtime to the current provider/model, configured runtime choices, or sender authority.

This is not just cosmetic. Runtime selection changes execution backend semantics. A chat user who is allowed to send model directives should not automatically be allowed to persist arbitrary runtime/harness overrides. The fix should validate the requested runtime before storing it, probably against the same runtime choices exposed to the picker plus configured agent policy, and reject/ignore unknown or unauthorized values. The execution path should also defensively refuse an unrecognized persisted override instead of treating it as a pinned harness id.

Smaller cleanup findings, non-blocking after the runtime gate is fixed:

  • extensions/anthropic/register.runtime.ts:81: CLAUDE_CLI_CANONICAL_DEFAULT_MODEL_REF slices CLAUDE_CLI_DEFAULT_MODEL_REF as if it always starts with claude-cli/. The allowlist mapping above has the correct startsWith guard; the default should use the same pattern to avoid silently producing a bad ref if the constant changes shape later.
  • src/config/schema.labels.ts:75: agents.defaults.embeddedHarness and agents.defaults.embeddedHarness.runtime both label as Default Agent Runtime, which makes flat config/help output ambiguous. Keep the parent distinct, e.g. Default Agent Harness, and the leaf as Default Agent Runtime.

Also current checks are still red (check, check-test-types, check-additional, checks-node-core-fast-support / related). Even if some are stale, this needs a fresh green run after the runtime validation fix.

@vincentkoc
vincentkoc force-pushed the fix/provider-runtime-selection-followup branch from 72e6a72 to f5c687e Compare April 24, 2026 23:54
@vincentkoc
vincentkoc merged commit aa27e27 into main Apr 24, 2026
71 checks passed
@vincentkoc
vincentkoc deleted the fix/provider-runtime-selection-followup branch April 24, 2026 23:56
Angfr95 pushed a commit to Angfr95/openclaw that referenced this pull request Apr 25, 2026
* fix(models): normalize provider runtime selection

* fix(models): reverse codex-only runtime migration

* fix(models): default runtime selection to pi

* fix(status): label model runtime clearly

* fix(status): align pi runtime label

* fix(plugins): align tool result middleware runtime naming

* fix(models): validate runtime overrides
ogt-redknie pushed a commit to ogt-redknie/OPENX that referenced this pull request May 2, 2026
* fix(models): normalize provider runtime selection

* fix(models): reverse codex-only runtime migration

* fix(models): default runtime selection to pi

* fix(status): label model runtime clearly

* fix(status): align pi runtime label

* fix(plugins): align tool result middleware runtime naming

* fix(models): validate runtime overrides
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 9, 2026
* fix(models): normalize provider runtime selection

* fix(models): reverse codex-only runtime migration

* fix(models): default runtime selection to pi

* fix(status): label model runtime clearly

* fix(status): align pi runtime label

* fix(plugins): align tool result middleware runtime naming

* fix(models): validate runtime overrides
globalcaos pushed a commit to globalcaos/tinkerclaw that referenced this pull request May 13, 2026
* fix(models): normalize provider runtime selection

* fix(models): reverse codex-only runtime migration

* fix(models): default runtime selection to pi

* fix(status): label model runtime clearly

* fix(status): align pi runtime label

* fix(plugins): align tool result middleware runtime naming

* fix(models): validate runtime overrides
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026
* fix(models): normalize provider runtime selection

* fix(models): reverse codex-only runtime migration

* fix(models): default runtime selection to pi

* fix(status): label model runtime clearly

* fix(status): align pi runtime label

* fix(plugins): align tool result middleware runtime naming

* fix(models): validate runtime overrides
jameslcowan pushed a commit to jameslcowan/openclaw that referenced this pull request Jun 2, 2026
* fix(models): normalize provider runtime selection

* fix(models): reverse codex-only runtime migration

* fix(models): default runtime selection to pi

* fix(status): label model runtime clearly

* fix(status): align pi runtime label

* fix(plugins): align tool result middleware runtime naming

* fix(models): validate runtime overrides
sablehead pushed a commit to sablehead/openclaw that referenced this pull request Jun 10, 2026
* fix(models): normalize provider runtime selection

* fix(models): reverse codex-only runtime migration

* fix(models): default runtime selection to pi

* fix(status): label model runtime clearly

* fix(status): align pi runtime label

* fix(plugins): align tool result middleware runtime naming

* fix(models): validate runtime overrides
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling channel: discord Channel integration: discord channel: mattermost Channel integration: mattermost commands Command implementations docs Improvements or additions to documentation extensions: anthropic extensions: codex extensions: tokenjuice Changes to the bundled tokenjuice extension gateway Gateway runtime maintainer Maintainer-authored PR size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants