fix(auto-reply): per-model runtime policy in memory-flush gates#86361
fix(auto-reply): per-model runtime policy in memory-flush gates#86361potterdigital wants to merge 1 commit into
Conversation
The memory-flush and preflight-compaction gates in agent-runner-memory.ts used `isCliProvider(provider, cfg)`, which only inspects `cliBackends` keys. A session with `provider="anthropic"` and a model whose `agents.defaults.models[<key>].agentRuntime.id` was pinned to "claude-cli" therefore reported "not CLI" and dispatched `runEmbeddedPiAgent` directly. The embedded harness then sent the claude-cli OAuth subscription token (sk-ant-oat01-...) to /v1/messages, which Anthropic rejects with 404 model_not_found because OAuth subscription tokens are only valid through the claude CLI subprocess. The 404 cascaded through every anthropic auth profile into 40-minute cooldowns, leaving the agent unable to reply until the cooldowns aged out. Resolve `resolveAgentHarnessPolicy().runtime` and combine with `isCliRuntimeAliasForProvider` so the gate also honors the per-model runtime policy, matching the contract used elsewhere in the dispatch path (`resolveCliRuntimeExecutionProvider`, `maybeCompactAgentHarnessSession`).
|
Thanks for the context here. I swept through the related work, and this is now duplicate or superseded. Keep open: the reported P1 memory-flush/runtime bug is valid, but this patch should not merge as-is because it adds a second partial CLI-runtime resolver instead of using the existing dispatch resolver. That leaves supported auth-profile and providerless matched-provider CLI routes on the embedded maintenance path, and an overlapping resolver-shaped PR is already open at #86224. Canonical path: Close this PR as superseded by #86224. So I’m closing this here and keeping the remaining discussion on #86224. Review detailsBest possible solution: Close this PR as superseded by #86224. Do we have a high-confidence way to reproduce the issue? Yes at source level: current main's gates only call isCliProvider on the canonical provider while the dispatch path resolves CLI runtime policy first. I did not rerun the live Slack scenario, but the PR body includes production gateway logs and regression-test reverse proof for the reported per-model path. Is this the best way to solve the issue? No, not as written: the change fixes the reported per-model slice but duplicates only part of the existing dispatch resolver. The narrower maintainable fix is to reuse resolveCliRuntimeExecutionProvider so auth-profile and providerless matched-provider routes stay aligned too. Security review: Security review cleared: The diff changes runtime gate logic and tests only; I found no new dependency, CI, script, secret-handling, or supply-chain surface. AGENTS.md: found and applied where relevant. What I checked:
Likely related people:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 73189e3ecb0c. |
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
ClawSweeper applied the proposed close for this PR.
|
Summary
runMemoryFlushIfNeededandrunPreflightCompactionIfNeededinsrc/auto-reply/reply/agent-runner-memory.tsgate the embedded memory-flush dispatch onisCliProvider(run.provider, cfg). That helper only inspectsagents.defaults.cliBackendskeys — it never resolves the per-modelagents.defaults.models[<key>].agentRuntime.idpolicy thatresolveCliRuntimeExecutionProviderandmaybeCompactAgentHarnessSessionalready honor.A session with
run.provider = "anthropic"andagents.defaults.models["anthropic/<model>"].agentRuntime.id = "claude-cli"reports "not CLI" at the gate and dispatchesrunEmbeddedPiAgentdirectly, even though every other dispatch path for the same session correctly routes through theclaudeCLI subprocess. The embedded harness then submits whatever Anthropic-prefixed auth profile it picks to/v1/messages. When the profile is a Claude Code OAuth subscription token (sk-ant-oat01-…) — common in user configs becauseopenclaw auth login --provider anthropicstores subscription tokens underprovider: anthropic, type: token— Anthropic rejects with404 model_not_foundby design (subscription tokens are only valid through theclaudeCLI subprocess). The 404 cascades through every anthropic auth profile into a 40-minute cooldown, and the agent stops replying on the affected lane until the cooldown ages out.This change resolves
resolveAgentHarnessPolicy().runtimeand combines the result withisCliRuntimeAliasForProviderin both gates, matching the contract thatresolveCliRuntimeExecutionProviderandmaybeCompactAgentHarnessSessionalready use.Scope note on the preflight change. The
runPreflightCompactionIfNeededgate has the same defectiveisCliProvider-only check, but its downstream call intomaybeCompactAgentHarnessSession(atsrc/agents/harness/selection.ts:471) already short-circuits viaisCliRuntimeAliasForProviderfor CLI-routed sessions today. That means the preflight bypass is currently protected by the downstream gate, not the local one. This PR fixes the local gate too as defense-in-depth: the defect class is identical, the fix is mechanically identical, and a future refactor of the downstream protection would otherwise re-expose the bypass silently. Reviewers preferring strict-scope can request the preflight change be split into its own PR — happy to do it that way.Real behavior proof
Behavior addressed: Slack DM lane wedged with
404 status code (no body)onanthropic/claude-opus-4-7[1m]after a long claude-cli turn, with all anthropic auth profiles cooled down for 40 minutes despite the model being pinned to theclaude-cliruntime viaagents.defaults.models["anthropic/<model>"].agentRuntime.id.Real environment tested: production gateway on Proxmox LXC (openclaw v2026.5.20, node v24.13.0) running live Slack DMs for a single user across 8 agents. Config has
agents.defaults.cliBackends.claude-cli.command = /usr/local/bin/claudeandagents.defaults.models["anthropic/claude-opus-4-7[1m]"].agentRuntime.id = "claude-cli".Exact steps or command run after this patch:
dist/model-runtime-aliases-*.jswith a temporary[DBG-CLI-RESOLVE]log line on every call toresolveCliRuntimeExecutionProviderso the dispatch resolver could be observed live.dist/agent-runner.runtime-*.js(the compiledagent-runner-memory.tsequivalent of this PR) on the production gateway and restarted viasystemctl --user restart openclaw-gateway.mainagent on opus-4-7[1m]) until the chat UI reported100% context used 1.3M / 1M— the exact precondition for the previous wedge.node scripts/run-vitest.mjs src/auto-reply/reply/agent-runner-memory.test.ts— 28/28 passing (26 existing + 2 new regression tests + a non-CLI-runtime negative test + a non-anthropic CLI-alias test added in response to adversarial review).git stashthe source change, re-run the 4 new tests → all fail loudly withrunEmbeddedPiAgentmock unexpectedly called.git stash pop→ all pass.pnpm tsgo:core— clean.pnpm check:changed— clean on all lanes except a pre-existingsrc/plugins/document-extractors.runtime.test.ts:53TS2554 onorigin/main(unrelated; reproducible without this branch).Evidence after fix:
a96335a9-…):[agent/embedded] embedded run agent end isError=true model=claude-opus-4-7[1m] provider=anthropic error=404 status code (no body)followed byauth profile failure state updated profile=sha256:… reason=model_not_found windowType=cooldownagainst three separate auth profile hashes within 5 seconds, thenFailoverError: No available auth profile for anthropicacross the fallback chain. Zero[DBG-CLI-RESOLVE]entries in the 53 seconds preceding the embedded run — confirming the failing run bypassedresolveCliRuntimeExecutionProviderentirely. SessionId rotated from the slack thread's stored03f422f8-…to a brand-new52a6e154-…, which matches the post-flushincrementCompactionCountrotation produced insiderunMemoryFlushIfNeeded.100% context used 1.3M / 1Mreported by the chat UI, user message sent. Gateway log shows three[DBG-CLI-RESOLVE] result="claude-cli"lines (preflight gate, memory-flush gate, model-fallback assertion) followed immediately by[agent/cli-backend] cli exec: provider=claude-cli model=claude-opus-4-7[1m] trigger=user useResume=true session=present resumeSession=4a30bca18050 reuse=reusable. No[agent/embedded]events. Noauth profile failureevents. Claude-cli session resumed and a 277,173ms / 1912-rawLine assistant turn completed cleanly, followed by additional user turns continuing to dispatch through the CLI subprocess.main(opus-4-7[1m]),forge,architect,researcher,gabe,gabe-voice,scribeall observed dispatching through[agent/cli-backend]for their cron heartbeats — every dispatch preceded by a[DBG-CLI-RESOLVE] … result="claude-cli"line — confirming the same gate fix protects every agent, not just the one used for the verification.AssertionError: expected runEmbeddedPiAgentMock not to be calledwhen the source change is reverted; both pass when the source change is applied.Observed result after fix: legitimate cron and slack-DM turns continue to resolve to
claude-clivia the existing dispatch path; the memory-flush and preflight-compaction gates now early-return for sessions whose per-model runtime policy pins them to a CLI runtime, so the embedded harness can no longer be invoked with OAuth subscription tokens through this code path. No 404 cascade has occurred on the production gateway since the fix was applied, including across the 1.3M-context boundary that previously wedged the lane.What was not tested:
codex/codex-clipaths inresolveAgentHarnessPolicy. The fix's added check isisCliRuntimeAliasForProvider, which evaluates against the staticCLI_RUNTIME_BY_PROVIDERmap. Codex entries arecli: falsein that map, so they never trip the new gate condition — but no codex-routed lane was available for live verification on the test gateway. The new tests cover anthropic + google CLI-alias positive cases and thepiruntime negative case to bound the behavior.runEmbeddedPiAgentpaths insrc/commitments/runtime.ts,src/hooks/llm-slug-generator.ts,src/crestodian/assistant.ts, andsrc/talk/agent-consult-runtime.ts. These use synthetic sessionKeys (temp:…,commitments:…) and do not currently route through the slack-DM autoreply path that triggered the production failure. They are out of scope for this PR but share the broader pattern of callingrunEmbeddedPiAgentwithout first consultingresolveCliRuntimeExecutionProvider; flagging as a separate audit item.Companion misconfiguration worth flagging (not in scope for this PR, addressed separately on the affected gateway): when
auth.profilesstores OAuth subscription tokens (sk-ant-oat01-…) under{ provider: "anthropic", type: "token" }rather than{ provider: "claude-cli", type: "oauth" }, the embedded harness — when reached — will pick those profiles and submit subscription tokens to/v1/messages, which 404s deterministically. The right user-facing fix is reclassification of those profiles (viaopenclaw config patchagainst the centralopenclaw.json auth.profilesand the per-agentauth-profiles.jsoncredential records, since both layers exist). This PR's gate fix prevents the embedded harness from reaching those profiles for CLI-routed models regardless of how the profiles are typed, eliminating the cascade as a side effect of the configuration issue.🤖 Generated with Claude Code