fix(utils): resolve catalog max_input_tokens in trim_messages for prefixed models - #50645

TLDR

Problem this solves:

• trim_messages only checked model in litellm.model_cost by exact key
• Provider-prefixed names like openai/gpt-4o and anthropic/claude-haiku-4-5 missed the bare-keyed cost map rows and skipped trimming

How it solves it:

• Keep the exact model in litellm.model_cost lookup first
• Fall back to lookup_catalog_model_info(model, None) when the exact key misses and max_tokens is unset
• Return messages unchanged when the model is still unknown or has no max_input_tokens

User Flow

Before: a Python SDK caller trimming messages for openai/gpt-4o or anthropic/claude-haiku-4-5 gets the untrimmed list back

1. They call trim_messages(messages, model="openai/gpt-4o") on a 301,412-token message list
2. trim_messages returns all 301,412 tokens untouched instead of trimming to 75% of the 128,000-token context window
3. Passing those messages to litellm.completion sends an over-limit request to the provider

After: the same call trims the conversation to ~75% of the model's context window

1. They call trim_messages(messages, model="openai/gpt-4o") on the same 301,412-token message list
2. trim_messages resolves the 128,000-token context window from the catalog and trims the messages down to 95,928 tokens

Relevant issues

Fixes #50639

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• Every automated review on my PR's latest commit has finished, and I have addressed or replied to each of its findings before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared reproduction command (from #50639, no API key required):

LITELLM_LOCAL_MODEL_COST_MAP=True uv run python -c '
import litellm
from litellm.utils import trim_messages

messages = [{"role": "user", "content": "This is a sentence. " * 60000}]
for model in ["gpt-4o", "openai/gpt-4o", "claude-haiku-4-5", "anthropic/claude-haiku-4-5"]:
    trimmed = trim_messages(messages, model=model)
    print(
        f"{model}: context window {litellm.get_model_info(model)[\"max_input_tokens\"]}, "
        f"tokens before {litellm.token_counter(model=model, messages=messages)}, "
        f"after {litellm.token_counter(model=model, messages=trimmed)}"
    )
'

Before (abee9c9)

1. Run the command above at abee9c945ba652f9c971eca8eabfca2600fdbfd1
2. Bare model names trim to ~75% of the window, while openai/gpt-4o and anthropic/claude-haiku-4-5 return all 301,412 tokens unchanged:

gpt-4o: context window 128000, tokens before 301412, after 95928
openai/gpt-4o: context window 128000, tokens before 301412, after 301412
claude-haiku-4-5: context window 200000, tokens before 300008, after 149996
anthropic/claude-haiku-4-5: context window 200000, tokens before 301412, after 301412

After (a0c6114)

1. Run the same command at a0c61146c0e8b78da88f832c5f7bea795833463e
2. Both bare and provider-prefixed model names trim to ~75% of their context windows, and uv run pytest tests/unit/test_utils.py -k "trim" -q passes (18 passed):

gpt-4o: context window 128000, tokens before 301412, after 95928
openai/gpt-4o: context window 128000, tokens before 301412, after 95928
claude-haiku-4-5: context window 200000, tokens before 300008, after 149996
anthropic/claude-haiku-4-5: context window 200000, tokens before 301412, after 149886

Type

🐛 Bug Fix

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR