Skip to content

fix(context): key MODEL_CACHE by provider/modelId to prevent collision (#14708)#14744

Closed
lailoo wants to merge 2 commits intoopenclaw:mainfrom
lailoo:fix/model-cache-provider-key-14708
Closed

fix(context): key MODEL_CACHE by provider/modelId to prevent collision (#14708)#14744
lailoo wants to merge 2 commits intoopenclaw:mainfrom
lailoo:fix/model-cache-provider-key-14708

Conversation

@lailoo
Copy link
Copy Markdown
Contributor

@lailoo lailoo commented Feb 12, 2026

Summary

  • Bug: MODEL_CACHE in context.ts keys entries by bare model ID only. When two providers register the same model ID (e.g. claude-opus-4-6 on both anthropic and a proxy), the last-loaded provider silently overwrites the first, returning the wrong context window.
  • Root cause: MODEL_CACHE.set(m.id, m.contextWindow) uses m.id as key without provider dimension. lookupContextTokens(modelId) has no provider parameter, so callers cannot disambiguate.
  • Fix: Key cache entries by provider/modelId, add optional provider parameter to lookupContextTokens, and pass provider from all 12 call sites where available.

Fixes #14708

Problem

context.ts builds a MODEL_CACHE map keyed by bare model ID:

MODEL_CACHE.set(m.id, m.contextWindow); // no provider dimension

When discoverModels().getAll() returns models with the same ID from different providers (e.g. anthropic/claude-opus-4-6 with 200k and my-proxy/claude-opus-4-6 with 128k), the second write silently overwrites the first. lookupContextTokens only accepts modelId, so callers have no way to get the correct value for a specific provider.

This affects session context management (agent-runner.ts, followup-runner.ts, cron/run.ts, etc.) — wrong context window leads to incorrect compaction thresholds and token budgets.

Before fix (reproduced on main via integration test):

pnpm vitest run src/agents/context.test.ts

✓ lookupContextTokens returns last-writer-wins for same model ID across providers
  → lookupContextTokens("claude-opus-4-6") = 128_000 (my-proxy's value)
  → anthropic's 200_000 is lost
✓ lookupContextTokens has no provider parameter (API limitation)
  → function.length ≤ 1

Test Files  1 passed (1)
Tests       2 passed (2)

Changes

  • src/agents/context.ts — Key MODEL_CACHE by provider/modelId; add bare model ID as first-writer-wins fallback; add optional provider param to lookupContextTokens
  • src/auto-reply/reply/agent-runner.ts — Pass providerUsed to lookupContextTokens
  • src/auto-reply/reply/agent-runner-memory.ts — Pass provider to resolveMemoryFlushContextWindowTokens
  • src/auto-reply/reply/directive-handling.persist.ts — Pass provider to lookupContextTokens
  • src/auto-reply/reply/followup-runner.ts — Pass provider to lookupContextTokens
  • src/auto-reply/reply/memory-flush.ts — Add optional provider param to resolveMemoryFlushContextWindowTokens
  • src/auto-reply/reply/model-selection.ts — Add optional provider param to resolveContextTokens
  • src/auto-reply/reply/get-reply-directives.ts — Pass provider to resolveContextTokens
  • src/auto-reply/status.ts — Pass provider to both lookupContextTokens calls
  • src/commands/sessions.ts — Pass resolved.provider where available
  • src/commands/status.summary.ts — Pass resolved.provider where available
  • src/commands/agent/session-store.ts — Pass providerUsed to lookupContextTokens
  • src/cron/isolated-agent/run.ts — Pass providerUsed to lookupContextTokens
  • src/gateway/session-utils.ts — Pass resolved.provider to lookupContextTokens
  • src/agents/context.test.ts — New regression tests
  • CHANGELOG.md — Add fix entry

After fix (verified on fix branch):

pnpm vitest run src/agents/context.test.ts

✓ returns correct context window per provider for same model ID
  → lookupContextTokens("claude-opus-4-6", "anthropic") = 200_000
  → lookupContextTokens("claude-opus-4-6", "my-proxy") = 128_000
✓ bare model ID fallback uses first-writer-wins
  → lookupContextTokens("claude-opus-4-6") = 200_000 (first provider wins)
✓ lookupContextTokens accepts optional provider parameter

Test Files  1 passed (1)
Tests       3 passed (3)

Design decisions

  • Backward compatible: provider param is optional. Callers without provider info still get a result via bare model ID fallback (first-writer-wins).
  • First-writer-wins for bare ID: When no provider is given, the first provider's value is used. This is better than last-writer-wins because it's deterministic and preserves the "primary" provider's value.
  • Not all call sites have provider: sessions.ts row iteration and status.summary.ts session rows don't have provider info — these fall back to bare model ID lookup, which is still better than the old behavior.

Test plan

  • Bug reproduced on main: last-writer-wins collision, no provider param (integration test)
  • Fix verified: provider-qualified lookups return correct values per provider
  • Fix verified: bare model ID fallback uses first-writer-wins
  • Fix verified: backward compatible (calling without provider still works)
  • All 10 existing memory-flush tests pass
  • TypeScript type check passes (pnpm tsgo — no errors in changed files)
  • Lint passes (pnpm lint)

Effect on User Experience

Before: When two providers register the same model ID with different context windows, the wrong context window is silently used. This causes incorrect compaction thresholds, potentially truncating conversation history too aggressively or not aggressively enough.

After: Each provider's context window is correctly preserved. Callers with provider info get the exact value; callers without provider info get a deterministic first-writer-wins fallback.

Greptile Overview

Greptile Summary

This change fixes collisions in MODEL_CACHE by qualifying cache keys with provider/modelId and extends lookupContextTokens to accept an optional provider so callers can request the correct context window when different providers expose the same model ID. Call sites across auto-reply/session/cron/gateway paths are updated to pass provider when available, and a new regression test is added to cover provider-qualified lookups and the bare-ID fallback behavior.

Confidence Score: 4/5

  • This PR is mostly safe to merge, but the new regression test is likely to be flaky due to timing-based initialization.
  • Core change is localized (cache keying + optional provider parameter) and call sites were updated consistently. The main concern is src/agents/context.test.ts using a fixed 50ms sleep to wait for async module initialization, which can fail nondeterministically in CI and block merges.
  • src/agents/context.test.ts

@openclaw-barnacle openclaw-barnacle bot added gateway Gateway runtime commands Command implementations agents Agent runtime and tooling size: S labels Feb 12, 2026
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

16 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Comment on lines +38 to +40
const { lookupContextTokens } = await import("./context.js");
await new Promise((r) => setTimeout(r, 50));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Racy async cache load

These tests rely on await new Promise((r) => setTimeout(r, 50)) after importing ./context.js, but lookupContextTokens() only kicks off the async load and never awaits it. On a slower CI machine (or under load), the cache may not be populated within 50ms, making the assertions flaky. Prefer awaiting a deterministic signal (e.g., export/await a loadPromise, or have lookupContextTokens optionally await initialization for tests) rather than sleeping.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/context.test.ts
Line: 38:40

Comment:
**Racy async cache load**

These tests rely on `await new Promise((r) => setTimeout(r, 50))` after importing `./context.js`, but `lookupContextTokens()` only kicks off the async load and never awaits it. On a slower CI machine (or under load), the cache may not be populated within 50ms, making the assertions flaky. Prefer awaiting a deterministic signal (e.g., export/await a `loadPromise`, or have `lookupContextTokens` optionally await initialization for tests) rather than sleeping.

How can I resolve this? If you propose a fix, please make it concise.

@openclaw-barnacle
Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle bot added stale Marked as stale due to inactivity and removed stale Marked as stale due to inactivity labels Feb 21, 2026
@steipete
Copy link
Copy Markdown
Contributor

Closing as AI-assisted stale-fix triage.

Linked issue #14708 ("Bug: MODEL_CACHE context window collision when same model ID exists across multiple providers") is currently CLOSED and was closed on 2026-02-13T02:39:25Z with state reason NOT_PLANNED.
Given that issue state, this fix PR is no longer needed in the active queue and is being closed as stale.

If the underlying bug is still reproducible on current main, please reopen this PR (or open a new focused fix PR) and reference both #14708 and #14744 for fast re-triage.

@steipete
Copy link
Copy Markdown
Contributor

Closed after AI-assisted stale-fix triage (closed issue duplicate/stale fix).

@steipete steipete closed this Feb 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling commands Command implementations gateway Gateway runtime size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: MODEL_CACHE context window collision when same model ID exists across multiple providers

2 participants