fix(context): key MODEL_CACHE by provider/modelId to prevent collision (#14708)#14744
fix(context): key MODEL_CACHE by provider/modelId to prevent collision (#14708)#14744lailoo wants to merge 2 commits intoopenclaw:mainfrom
Conversation
src/agents/context.test.ts
Outdated
| const { lookupContextTokens } = await import("./context.js"); | ||
| await new Promise((r) => setTimeout(r, 50)); | ||
|
|
There was a problem hiding this comment.
Racy async cache load
These tests rely on await new Promise((r) => setTimeout(r, 50)) after importing ./context.js, but lookupContextTokens() only kicks off the async load and never awaits it. On a slower CI machine (or under load), the cache may not be populated within 50ms, making the assertions flaky. Prefer awaiting a deterministic signal (e.g., export/await a loadPromise, or have lookupContextTokens optionally await initialization for tests) rather than sleeping.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/context.test.ts
Line: 38:40
Comment:
**Racy async cache load**
These tests rely on `await new Promise((r) => setTimeout(r, 50))` after importing `./context.js`, but `lookupContextTokens()` only kicks off the async load and never awaits it. On a slower CI machine (or under load), the cache may not be populated within 50ms, making the assertions flaky. Prefer awaiting a deterministic signal (e.g., export/await a `loadPromise`, or have `lookupContextTokens` optionally await initialization for tests) rather than sleeping.
How can I resolve this? If you propose a fix, please make it concise.bfc1ccb to
f92900f
Compare
|
This pull request has been automatically marked as stale due to inactivity. |
|
Closing as AI-assisted stale-fix triage. Linked issue #14708 ("Bug: MODEL_CACHE context window collision when same model ID exists across multiple providers") is currently CLOSED and was closed on 2026-02-13T02:39:25Z with state reason NOT_PLANNED. If the underlying bug is still reproducible on current main, please reopen this PR (or open a new focused fix PR) and reference both #14708 and #14744 for fast re-triage. |
|
Closed after AI-assisted stale-fix triage (closed issue duplicate/stale fix). |
Summary
MODEL_CACHEincontext.tskeys entries by bare model ID only. When two providers register the same model ID (e.g.claude-opus-4-6on bothanthropicand a proxy), the last-loaded provider silently overwrites the first, returning the wrong context window.MODEL_CACHE.set(m.id, m.contextWindow)usesm.idas key without provider dimension.lookupContextTokens(modelId)has noproviderparameter, so callers cannot disambiguate.provider/modelId, add optionalproviderparameter tolookupContextTokens, and pass provider from all 12 call sites where available.Fixes #14708
Problem
context.tsbuilds aMODEL_CACHEmap keyed by bare model ID:When
discoverModels().getAll()returns models with the same ID from different providers (e.g.anthropic/claude-opus-4-6with 200k andmy-proxy/claude-opus-4-6with 128k), the second write silently overwrites the first.lookupContextTokensonly acceptsmodelId, so callers have no way to get the correct value for a specific provider.This affects session context management (
agent-runner.ts,followup-runner.ts,cron/run.ts, etc.) — wrong context window leads to incorrect compaction thresholds and token budgets.Before fix (reproduced on main via integration test):
Changes
src/agents/context.ts— KeyMODEL_CACHEbyprovider/modelId; add bare model ID as first-writer-wins fallback; add optionalproviderparam tolookupContextTokenssrc/auto-reply/reply/agent-runner.ts— PassproviderUsedtolookupContextTokenssrc/auto-reply/reply/agent-runner-memory.ts— PassprovidertoresolveMemoryFlushContextWindowTokenssrc/auto-reply/reply/directive-handling.persist.ts— PassprovidertolookupContextTokenssrc/auto-reply/reply/followup-runner.ts— PassprovidertolookupContextTokenssrc/auto-reply/reply/memory-flush.ts— Add optionalproviderparam toresolveMemoryFlushContextWindowTokenssrc/auto-reply/reply/model-selection.ts— Add optionalproviderparam toresolveContextTokenssrc/auto-reply/reply/get-reply-directives.ts— PassprovidertoresolveContextTokenssrc/auto-reply/status.ts— Passproviderto bothlookupContextTokenscallssrc/commands/sessions.ts— Passresolved.providerwhere availablesrc/commands/status.summary.ts— Passresolved.providerwhere availablesrc/commands/agent/session-store.ts— PassproviderUsedtolookupContextTokenssrc/cron/isolated-agent/run.ts— PassproviderUsedtolookupContextTokenssrc/gateway/session-utils.ts— Passresolved.providertolookupContextTokenssrc/agents/context.test.ts— New regression testsCHANGELOG.md— Add fix entryAfter fix (verified on fix branch):
Design decisions
providerparam is optional. Callers without provider info still get a result via bare model ID fallback (first-writer-wins).sessions.tsrow iteration andstatus.summary.tssession rows don't have provider info — these fall back to bare model ID lookup, which is still better than the old behavior.Test plan
pnpm tsgo— no errors in changed files)pnpm lint)Effect on User Experience
Before: When two providers register the same model ID with different context windows, the wrong context window is silently used. This causes incorrect compaction thresholds, potentially truncating conversation history too aggressively or not aggressively enough.
After: Each provider's context window is correctly preserved. Callers with provider info get the exact value; callers without provider info get a deterministic first-writer-wins fallback.
Greptile Overview
Greptile Summary
This change fixes collisions in
MODEL_CACHEby qualifying cache keys withprovider/modelIdand extendslookupContextTokensto accept an optionalproviderso callers can request the correct context window when different providers expose the same model ID. Call sites across auto-reply/session/cron/gateway paths are updated to pass provider when available, and a new regression test is added to cover provider-qualified lookups and the bare-ID fallback behavior.Confidence Score: 4/5
src/agents/context.test.tsusing a fixed 50ms sleep to wait for async module initialization, which can fail nondeterministically in CI and block merges.