-
Notifications
You must be signed in to change notification settings - Fork 2k
Performance Overhead of Mounted Multi-Server Proxy #3466
Copy link
Copy link
Closed
Labels
bugSomething isn't working. Reports of errors, unexpected behavior, or broken functionality.Something isn't working. Reports of errors, unexpected behavior, or broken functionality.httpRelated to HTTP transport, networking, or web server functionality.Related to HTTP transport, networking, or web server functionality.potential-duplicateBot-suggested duplicate awaiting human review. Auto-closes after 3 days if unchallenged.Bot-suggested duplicate awaiting human review. Auto-closes after 3 days if unchallenged.serverRelated to FastMCP server implementation or server-side functionality.Related to FastMCP server implementation or server-side functionality.
Metadata
Metadata
Assignees
Labels
bugSomething isn't working. Reports of errors, unexpected behavior, or broken functionality.Something isn't working. Reports of errors, unexpected behavior, or broken functionality.httpRelated to HTTP transport, networking, or web server functionality.Related to HTTP transport, networking, or web server functionality.potential-duplicateBot-suggested duplicate awaiting human review. Auto-closes after 3 days if unchallenged.Bot-suggested duplicate awaiting human review. Auto-closes after 3 days if unchallenged.serverRelated to FastMCP server implementation or server-side functionality.Related to FastMCP server implementation or server-side functionality.
Description
Hello, I am not sure if this is a bug or intended.
I am on the latest FastMCP version: 3.1.0
My current setup is to mount a multi-server proxy on an existing FastAPI instance.
The proxy is created similar to this example: https://gofastmcp.com/servers/providers/proxy#multi-server-proxies
Then the proxy is mounted as a
stateless-httphttp app.What I observe during my performance tests is that, there seems to be a huge performance overhead when making direct tool calls against the proxy.
100 Requests.
Proxy: Average of 15s
Direct to MCP: Average of 80ms
On analysis, it seems that:
When creating a proxy with
create_proxy(), the default_create_client_factory()is used.Since
client_factoryis invoked on every mcp method call on the proxy server, there is a new ProxyClient being created withbase_client.new().https://github.com/PrefectHQ/fastmcp/blob/main/src/fastmcp/server/providers/proxy.py#L671-L678
This causes
connect_sessionto be called multiple times, once on list_tools and another on the actual tool call. Resulting in multiple mcp initialization handshakes.I do think this might be intended to prevent context mixing.
However, since I am intending to go with a
stateless-httpproxy server.Would overriding the
client_factoryhere to return the same client make sense?This approach dramatically improves the performance from 15s to 5s.
Or is there a better approach I can do to avoid the overhead caused by mcp session initialization when running the proxy in stateless mode?
Version Information