Python: add progressive MCP discovery/dispatch mode for large MCP servers #5821

Feature request

The Python SDK has good MCP primitives today: MCPStreamableHTTPTool connects to MCP servers, loads remote tools, filters with allowed_tools, exposes remote tools as FunctionTools, parses MCP results, raises on isError, and injects OTel metadata.

For large production MCP servers, it would be useful to also support a progressive model-facing MCP mode:

• expose a small stable discovery tool, e.g. list_mcp_tools(server?: str)
• expose a small stable dispatch tool, e.g. call_mcp(server: str, tool: str, arguments: dict)
• back both with SDK MCP transports/sessions/result parsing/OTel
• resolve call_mcp against the SDK-loaded filtered function list before execution
• keep allowed_tools / approval / remote-name metadata semantics intact
• avoid exposing MCPStreamableHTTPTool.call_tool() raw to the model

Why

Direct first-class MCP exposure is excellent when the remote surface is small. With larger MCP servers or multiple servers, exposing every remote schema up front can add significant prompt/tool-definition overhead and can force skills/prompts to be coupled to every remote tool shape.

A progressive mode lets the model discover exact remote tools and schemas only when needed, while preserving SDK-owned MCP transport, telemetry, result parsing, and error behavior.

Current workaround

Applications can implement this as a thin compatibility adapter by creating SDK MCPStreamableHTTPTool instances internally, reading their filtered .functions, and exposing local FunctionTools for list_mcp_tools / call_mcp. That works, but it duplicates policy-sensitive behavior that would be better owned by the SDK.

Suggested API shape

One possible shape:

mcp = MCPStreamableHTTPTool(..., allowed_tools=[...])
progressive_tools = mcp.as_progressive_tools(
    list_tool_name="list_mcp_tools",
    call_tool_name="call_mcp",
)
agent = Agent(..., tools=[*progressive_tools])

Or an agent/tool option such as exposure_mode="progressive".

The important behavior is that the model sees the small progressive surface, while the SDK still owns connection lifecycle, allowed tool filtering, approval handling, MCP result parsing, exceptions, and OTel propagation.