Conversation
…d connections Every call_tool through a proxy was triggering _list_tools() to resolve the tool by name, opening a full MCP session just for the lookup, then opening a second session for the actual execution. This caches component lists on the ProxyProvider with a configurable TTL (default 300s), cutting backend handshakes in half for repeated calls.
jlowin
added
enhancement
marvin-context-protocol
Bot
added
the
provider
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 79e086f867
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| cache = self._tools_cache | ||
| if cache is None or not cache.is_fresh(self._cache_ttl): | ||
| await self._list_tools() | ||
| cache = self._tools_cache |
There was a problem hiding this comment.
_get_tool now serves lookups from a single provider-wide cache and skips _list_tools() while the TTL is fresh, so client_factory is not consulted per request. This breaks valid setups where client_factory is context-dependent (for example StatefulProxyClient.new_stateful or factories that pick a backend/token from request context): the first request’s catalog is reused for subsequent sessions, causing false "not found" results (or stale component exposure) until expiry. The same pattern is repeated for resources/templates/prompts, so the cache needs to be keyed by request/session identity or disabled for dynamic factories.
Useful? React with 👍 / 👎.
Test Failure AnalysisSummary: The Windows CI job failed due to a timing race in Root Cause: In The sibling test Suggested Solution: Add # In test_cache_expires_after_ttl
original_ts = provider._tools_cache.timestamp
time.sleep(0.01) # allow monotonic clock to advance (especially on Windows)
await provider._get_tool("greet")
assert provider._tools_cache.timestamp > original_tsFile to change: Detailed AnalysisFailing test ( async def test_cache_expires_after_ttl(self, fastmcp_server):
"""After TTL expires, _get_tool should re-fetch from the backend."""
provider = ProxyProvider(
lambda: ProxyClient(FastMCPTransport(fastmcp_server)),
cache_ttl=0.0,
)
# Warm the cache
await provider._list_tools()
assert provider._tools_cache is not None
original_ts = provider._tools_cache.timestamp
await provider._get_tool("greet")
assert provider._tools_cache.timestamp > original_ts # <-- fails on WindowsPassing sibling test (uses async def test_list_tools_refreshes_cache(self, fastmcp_server):
...
# Tiny sleep so monotonic clock advances
time.sleep(0.01)
await provider._list_tools()
assert provider._tools_cache.timestamp > first_tsLog excerpt: Only 1 test failed (4804 passed). The fix is a one-line Related Files
|
Ports the proxy-component cache from FastMCP 3.2.0 PR PrefectHQ#3479 onto the v2.14.4 base. ProxyToolManager/ProxyResourceManager/ProxyPromptManager now memoize their get_*() results for cache_ttl seconds (default 300s, set via new optional cache_ttl kwarg on each __init__). The 2.x proxy layer lives in src/fastmcp/server/proxy.py as three separate managers (vs 3.x's single ProxyProvider), so the upstream 3.x logic is adapted class-by-class while preserving the public constructor signatures. Since ToolManager.get_tool(key) calls get_tools() and indexes the returned dict, caching get_tools() transparently accelerates per-call dispatch — which is the mechanism causing the ~14.5s per-tool-call regression on the Orbit agent path. New test file tests/server/test_proxy_cache.py covers cache hit/miss, TTL expiry, cache_ttl=0 disables, invalidate_cache(), isolation between tool/resource/template/prompt managers, and that get_tool(key) lookups reuse the cached list. See ORBIT_FORK.md for provenance and deprecation criteria.
1 participant
Closes #3466
Every
call_toolthrough a proxy was opening two full MCP sessions to the backend: one to resolve the tool by name (via_get_tool→_list_tools()), and another to actually execute it inProxyTool.run(). For stateless HTTP proxies, each session means a fresh TCP connection and MCP initialization handshake — so 100 tool calls produced 200 handshakes.ProxyProvidernow caches component lists (tools, resources, templates, prompts) on the instance with a configurable TTL (default 300s). When_get_toolis called during tool resolution, it resolves from the cache instead of opening a backend connection. The cache refreshes whenever an explicit_list_*call is made.For the remaining per-call handshake, the docs now show how to opt into session reuse for stateless backends — return the same client instance from the factory to skip the MCP initialization on every call: