Skip to content

[fix] Include response schema in model cache keys - #10621

Merged
sannya-singal merged 4 commits into
agno-agi:mainfrom
charan-rathore:fix/cache-response-schema
Oct 7, 2026
Merged

sannya-singal merged 4 commits into
agno-agi:mainfrom
charan-rathore:fix/cache-response-schema

Conversation

@charan-rathore

Copy link
Copy Markdown
Contributor

Summary

_get_model_cache_key currently serializes a Pydantic response-format class using its class name. Two different Output classes with different fields therefore generate the same key for identical messages and model settings. With cache_response=True, the second call can reuse a response generated for the wrong output schema.

Serialize Pydantic response-format classes using model_json_schema() instead. This includes nested field definitions in the cache key while leaving other class serialization unchanged. This follows up on #7155, which made class inputs serializable but used the class name as the cache identity.

Type of change

  • Bug fix

Checklist

  • Code complies with style guidelines
  • Self-review completed
  • Documentation updated (regression test docstrings; no user-facing docs change)
  • Tests added/updated

Duplicate and AI-Generated PR Check

  • I have searched existing open pull requests and confirmed that no other PR already addresses this issue
  • This PR was generated with AI assistance and reviewed with focused tests.

Additional Notes

The new same-name schema tests cover streaming and non-streaming keys; equivalent schemas keep the same key. A mocked HTTP regression verifies that sync and async requests do not replay another output schema's cached response. pytest -q libs/agno/tests/unit/models/test_cache_key_serialization.py: 10 passed. ruff check, ruff format --check, and git diff --check pass. The broader model suite was not run locally because optional provider packages are missing; no provider call is needed for these focused tests.

@chrikrah chrikrah left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@charan-rathore I would merge this. Reverting models/base.py with your tests kept fails 4 of the 10, and the change touches only the type branch of _cache_default.

Two other open pull requests edit the same cache_data dict: #10137 adds tools and #10525 adds images. All three merge cleanly in any pair, and merged together they give distinct keys for schema, tools and images.

Evidence at 853b358, agno from the branch on PYTHONPATH, Python 3.12.3:

$ pytest tests/unit/models/test_cache_key_serialization.py -q
10 passed in 2.59s
# models/base.py from origin/main, tests kept
4 failed, 6 passed     test_same_named_response_formats_have_distinct_cache_keys[False|True]
                       test_response_cache_does_not_replay_another_same_named_schema[False|True]

$ git merge-tree --write-tree refs/pr/10621 refs/pr/10525   # and each other pair
clean

$ python probe_keys.py      # tree with all three merged
schema differs -> distinct: True
tools differ   -> distinct: True
images differ  -> distinct: True

non-blocking: audio, files and videos still share a key after all three. That is outside this change and filed as #10694.

@kausmeows, would you want the three merged in one order, or does it not matter to you?

@charan-rathore

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review and for checking the tests against the reverted implementation. The evidence and compatibility checks are really helpful.

@charan-rathore

Copy link
Copy Markdown
Contributor Author

Thanks for the review. This is ready whenever you are. If the other two PRs touching the same cache_data dict land first, I'm happy to rebase.

@sannya-singal sannya-singal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @charan-rathore, found some gaps, fixed in df93c79.

@sannya-singal
sannya-singal merged commit e7f7bd3 into agno-agi:main Oct 7, 2026
11 checks passed
@sannya-singal sannya-singal self-assigned this Oct 7, 2026
@charan-rathore
charan-rathore deleted the fix/cache-response-schema branch October 7, 2026 11:12
charan-rathore added a commit to charan-rathore/charan-rathore that referenced this pull request Oct 7, 2026
@kausmeows kausmeows mentioned this pull request Oct 8, 2026
ysolanky added a commit that referenced this pull request Oct 8, 2026
# Changelog

## New Features:

- **Conversation Compaction**: `Agent(compaction=True)` keeps long
sessions inside the context window by folding older turns into a
summary. Stored messages are never rewritten: a `CompactionRecord` in
the new `agno_compactions` table drives a derived model payload, so
removing the record restores the full conversation. Tune with
`Compaction(model=..., compact_at_tokens=..., uncompacted_runs=...)`,
fold on demand with `agent.compact()` / `acompact()`, and inspect
`run.compaction`. With `searchable=True` the agent gets a
`search_compacted_history` tool, a regex grep over the archived
transcript. See
[cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/02_agents/03_context_management/compaction/compaction.py).
(#9873)
- **Codex External Agent**: Added `CodexAgent`, an adapter for OpenAI
Codex built on the `openai-codex` SDK, alongside the Claude Agent SDK,
LangGraph, DSPy and Antigravity adapters. Codex runs its own agent loop;
the adapter translates its notifications into Agno run and tool-call
events, maps each Agno session to one Codex thread (persisted when a
`db` is set), and exposes `sandbox`, `approval_mode`,
`reasoning_effort`, `output_schema` and MCP config. Works standalone and
through AgentOS. See
[cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/frameworks/codex/codex_basic.py).
(#10901)
- **HyDE Query Transform**: Added `QueryTransformer` as a new retrieval
hook and `HyDE` as its first implementation: `Knowledge(vector_db=...,
query_transformer=HyDE())` searches with a hypothetical answer instead
of the raw question, while rerankers still score against the original
query. `HyDE` uses its own `model`, then the calling agent or team's
model, then the framework default. See
[cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/07_knowledge/02_building_blocks/12_hyde_query_transformer.py).
(#10527)
- **Browser Origin Policy**: Added `AgentOS(cors=CORSConfig(origins=...,
origin_regex=..., merge_base_app=...))` and opt-in
`PublicSurface(enforce_browser_origins=True)`. One origin policy is now
shared by CORS preflights, public run and cancel admission, auth error
headers, workflow WebSockets and MCP aliases, and CORS is the outermost
middleware so error responses carry consistent headers.
`cors_allowed_origins` keeps its existing behaviour. See
[cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/05_agent_os/27_public_pages/browser_origins.py).
(#10869)
- **Configurable Follow-ups**: `followups: bool | FollowupConfig` on
Agent and Team. `FollowupConfig` sets a count range (`num_followups` or
`min_followups` / `max_followups`), a separate model and custom
instructions that go only to the follow-up call. The default prompt now
stays within the answer's boundaries and no longer re-offers a request
the answer declined. Existing `num_followups` callers keep getting
exactly N suggestions. See
[cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/02_agents/14_advanced/followup_instructions.py).
(#10087)
- **Heabsy**: Added `Heabsy` as an OpenAI-compatible model provider,
also resolvable as `"heabsy:<model>"`. See
[cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/90_models/heabsy/basic.py).
(#10828)
- **FlexAI**: Added `FlexAI` as an OpenAI-compatible model provider for
open-weight models, also resolvable as `"flexai:<model>"`. See
[cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/90_models/flexai/basic.py).
(#10871)
- **SixtyDBTools**: New toolkit for the 60db hosted speech API: discover
workspace voices and generate speech as WAV audio artifacts. See
[cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/91_tools/sixtydb_tools.py).
(#10786)

## Improvements:

- **Page Sync Logs**: Page synchronization now logs lock waits,
discovery, progress every 25 pages and final counts, so long syncs show
progress without a custom callback. Added an AgentOS cookbook that syncs
documentation into PostgreSQL-backed knowledge pages and exposes them
through the agent's filesystem. See
[cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/05_agent_os/27_filesystem/knowledge_fs.py).
(#10868)
- **OpenAIResponses & OpenRouterResponses**: Recognise GPT-6 (OpenAI)
and GPT-5/6 (OpenRouter) as reasoning models, so reasoning-state
chaining and encrypted reasoning replay apply and tool-call follow-ups
no longer fail with missing reasoning-item errors. (#10830)
- **PerplexitySearch**: Requests now send an `X-Pplx-Integration: agno`
header so Perplexity can attribute traffic from the integration.
(#10827)

## Bug Fixes:

- **Approvals**: Each `@approval` tool call in a run now gets its own
approval record, for agents and teams. Previously a second pause in the
same run created no record and the next continue ran the new call
against the approval resolved for the first one. (#10812)
- **Model Response Cache**: Pydantic `output_schema` classes are keyed
by their JSON schema instead of their class name, so two schemas with
the same name but different fields no longer share a cached response.
(#10621)
- **Usage Metrics**: `MessageMetrics.cache_write_tokens` is now
populated from OpenAI Chat Completions and Responses usage (#10313) and
from LiteLLM's `prompt_tokens_details` (#10476). Previously it was
always zero.
- **JsonDb**: Tables are serialized to a staging file and published with
`os.replace`, so a failed write leaves the last successfully saved table
readable instead of a truncated file. (#10794)
- **MongoDb & MongoVectorDb**: `session_name`, `search_content` and
keyword-search filters are regex-escaped, so a value like `(draft` no
longer returns HTTP 500 from `GET /sessions` and `GET /memories`, and
`v1.2` no longer matches `v1x2`. (#10709)
- **Tool Schemas**: `Annotated[T, ...]` hints are unwrapped to `T`
before JSON schema generation instead of falling through to an empty
object schema. (#10858)
- **SQLTools**: Connection URLs are built with `URL.create`, so a
password like `p@ssword` is no longer split into password `p` and host
`ssword@...`. (#10800)
- **FileTools**: `replace_file_chunk` rejects negative, reversed and
out-of-bounds line ranges with an error and leaves the file untouched; a
valid replacement preserves the trailing newline. (#10798)
- **GoogleCalendarTools**: `find_available_slots` compares the full slot
end against closing time on its start date, so a 45 minute slot is no
longer offered at 16:30 for a 17:00 close and slots cannot cross
midnight. (#10878)
- **BaiduSearchTools, Searxng & SpiderTools**: An explicit
`max_results=0` / `fixed_max_results=0` now means no results instead of
falling back to the default. (#10877)
- **Readers**: `TextReader`, `MarkdownReader` (#10779) and `JSONReader`
(#10634) accept streams whose `name` is an integer or `None`, such as
descriptor-backed temporary files, instead of returning nothing or
raising.
- **Cookbooks**: Fixed the `DynamoDb` import path in the DynamoDB
storage README. (#10788)

---------

Co-authored-by: Yash Pratap Solanky <[email protected]>
Co-authored-by: ysolanky <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants