Repository navigation
[fix] Include response schema in model cache keys - #10621
Conversation
chrikrah
left a comment
There was a problem hiding this comment.
@charan-rathore I would merge this. Reverting models/base.py with your tests kept fails 4 of the 10, and the change touches only the type branch of _cache_default.
Two other open pull requests edit the same cache_data dict: #10137 adds tools and #10525 adds images. All three merge cleanly in any pair, and merged together they give distinct keys for schema, tools and images.
Evidence at 853b358, agno from the branch on PYTHONPATH, Python 3.12.3:
$ pytest tests/unit/models/test_cache_key_serialization.py -q
10 passed in 2.59s
# models/base.py from origin/main, tests kept
4 failed, 6 passed test_same_named_response_formats_have_distinct_cache_keys[False|True]
test_response_cache_does_not_replay_another_same_named_schema[False|True]
$ git merge-tree --write-tree refs/pr/10621 refs/pr/10525 # and each other pair
clean
$ python probe_keys.py # tree with all three merged
schema differs -> distinct: True
tools differ -> distinct: True
images differ -> distinct: True
non-blocking: audio, files and videos still share a key after all three. That is outside this change and filed as #10694.
@kausmeows, would you want the three merged in one order, or does it not matter to you?
|
Thanks for the thorough review and for checking the tests against the reverted implementation. The evidence and compatibility checks are really helpful. |
|
Thanks for the review. This is ready whenever you are. If the other two PRs touching the same cache_data dict land first, I'm happy to rebase. |
sannya-singal
left a comment
There was a problem hiding this comment.
thanks @charan-rathore, found some gaps, fixed in df93c79.
# Changelog ## New Features: - **Conversation Compaction**: `Agent(compaction=True)` keeps long sessions inside the context window by folding older turns into a summary. Stored messages are never rewritten: a `CompactionRecord` in the new `agno_compactions` table drives a derived model payload, so removing the record restores the full conversation. Tune with `Compaction(model=..., compact_at_tokens=..., uncompacted_runs=...)`, fold on demand with `agent.compact()` / `acompact()`, and inspect `run.compaction`. With `searchable=True` the agent gets a `search_compacted_history` tool, a regex grep over the archived transcript. See [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/02_agents/03_context_management/compaction/compaction.py). (#9873) - **Codex External Agent**: Added `CodexAgent`, an adapter for OpenAI Codex built on the `openai-codex` SDK, alongside the Claude Agent SDK, LangGraph, DSPy and Antigravity adapters. Codex runs its own agent loop; the adapter translates its notifications into Agno run and tool-call events, maps each Agno session to one Codex thread (persisted when a `db` is set), and exposes `sandbox`, `approval_mode`, `reasoning_effort`, `output_schema` and MCP config. Works standalone and through AgentOS. See [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/frameworks/codex/codex_basic.py). (#10901) - **HyDE Query Transform**: Added `QueryTransformer` as a new retrieval hook and `HyDE` as its first implementation: `Knowledge(vector_db=..., query_transformer=HyDE())` searches with a hypothetical answer instead of the raw question, while rerankers still score against the original query. `HyDE` uses its own `model`, then the calling agent or team's model, then the framework default. See [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/07_knowledge/02_building_blocks/12_hyde_query_transformer.py). (#10527) - **Browser Origin Policy**: Added `AgentOS(cors=CORSConfig(origins=..., origin_regex=..., merge_base_app=...))` and opt-in `PublicSurface(enforce_browser_origins=True)`. One origin policy is now shared by CORS preflights, public run and cancel admission, auth error headers, workflow WebSockets and MCP aliases, and CORS is the outermost middleware so error responses carry consistent headers. `cors_allowed_origins` keeps its existing behaviour. See [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/05_agent_os/27_public_pages/browser_origins.py). (#10869) - **Configurable Follow-ups**: `followups: bool | FollowupConfig` on Agent and Team. `FollowupConfig` sets a count range (`num_followups` or `min_followups` / `max_followups`), a separate model and custom instructions that go only to the follow-up call. The default prompt now stays within the answer's boundaries and no longer re-offers a request the answer declined. Existing `num_followups` callers keep getting exactly N suggestions. See [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/02_agents/14_advanced/followup_instructions.py). (#10087) - **Heabsy**: Added `Heabsy` as an OpenAI-compatible model provider, also resolvable as `"heabsy:<model>"`. See [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/90_models/heabsy/basic.py). (#10828) - **FlexAI**: Added `FlexAI` as an OpenAI-compatible model provider for open-weight models, also resolvable as `"flexai:<model>"`. See [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/90_models/flexai/basic.py). (#10871) - **SixtyDBTools**: New toolkit for the 60db hosted speech API: discover workspace voices and generate speech as WAV audio artifacts. See [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/91_tools/sixtydb_tools.py). (#10786) ## Improvements: - **Page Sync Logs**: Page synchronization now logs lock waits, discovery, progress every 25 pages and final counts, so long syncs show progress without a custom callback. Added an AgentOS cookbook that syncs documentation into PostgreSQL-backed knowledge pages and exposes them through the agent's filesystem. See [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/05_agent_os/27_filesystem/knowledge_fs.py). (#10868) - **OpenAIResponses & OpenRouterResponses**: Recognise GPT-6 (OpenAI) and GPT-5/6 (OpenRouter) as reasoning models, so reasoning-state chaining and encrypted reasoning replay apply and tool-call follow-ups no longer fail with missing reasoning-item errors. (#10830) - **PerplexitySearch**: Requests now send an `X-Pplx-Integration: agno` header so Perplexity can attribute traffic from the integration. (#10827) ## Bug Fixes: - **Approvals**: Each `@approval` tool call in a run now gets its own approval record, for agents and teams. Previously a second pause in the same run created no record and the next continue ran the new call against the approval resolved for the first one. (#10812) - **Model Response Cache**: Pydantic `output_schema` classes are keyed by their JSON schema instead of their class name, so two schemas with the same name but different fields no longer share a cached response. (#10621) - **Usage Metrics**: `MessageMetrics.cache_write_tokens` is now populated from OpenAI Chat Completions and Responses usage (#10313) and from LiteLLM's `prompt_tokens_details` (#10476). Previously it was always zero. - **JsonDb**: Tables are serialized to a staging file and published with `os.replace`, so a failed write leaves the last successfully saved table readable instead of a truncated file. (#10794) - **MongoDb & MongoVectorDb**: `session_name`, `search_content` and keyword-search filters are regex-escaped, so a value like `(draft` no longer returns HTTP 500 from `GET /sessions` and `GET /memories`, and `v1.2` no longer matches `v1x2`. (#10709) - **Tool Schemas**: `Annotated[T, ...]` hints are unwrapped to `T` before JSON schema generation instead of falling through to an empty object schema. (#10858) - **SQLTools**: Connection URLs are built with `URL.create`, so a password like `p@ssword` is no longer split into password `p` and host `ssword@...`. (#10800) - **FileTools**: `replace_file_chunk` rejects negative, reversed and out-of-bounds line ranges with an error and leaves the file untouched; a valid replacement preserves the trailing newline. (#10798) - **GoogleCalendarTools**: `find_available_slots` compares the full slot end against closing time on its start date, so a 45 minute slot is no longer offered at 16:30 for a 17:00 close and slots cannot cross midnight. (#10878) - **BaiduSearchTools, Searxng & SpiderTools**: An explicit `max_results=0` / `fixed_max_results=0` now means no results instead of falling back to the default. (#10877) - **Readers**: `TextReader`, `MarkdownReader` (#10779) and `JSONReader` (#10634) accept streams whose `name` is an integer or `None`, such as descriptor-backed temporary files, instead of returning nothing or raising. - **Cookbooks**: Fixed the `DynamoDb` import path in the DynamoDB storage README. (#10788) --------- Co-authored-by: Yash Pratap Solanky <[email protected]> Co-authored-by: ysolanky <[email protected]>
Summary
_get_model_cache_keycurrently serializes a Pydantic response-format class using its class name. Two differentOutputclasses with different fields therefore generate the same key for identical messages and model settings. Withcache_response=True, the second call can reuse a response generated for the wrong output schema.Serialize Pydantic response-format classes using
model_json_schema()instead. This includes nested field definitions in the cache key while leaving other class serialization unchanged. This follows up on #7155, which made class inputs serializable but used the class name as the cache identity.Type of change
Checklist
Duplicate and AI-Generated PR Check
Additional Notes
The new same-name schema tests cover streaming and non-streaming keys; equivalent schemas keep the same key. A mocked HTTP regression verifies that sync and async requests do not replay another output schema's cached response.
pytest -q libs/agno/tests/unit/models/test_cache_key_serialization.py: 10 passed.ruff check,ruff format --check, andgit diff --checkpass. The broader model suite was not run locally because optional provider packages are missing; no provider call is needed for these focused tests.