Python: MCPStreamableHTTPTool.connect() never sends a header_provider's headers, so any headers_env/header_provider-authenticated server 401s on connect #7841

Summary

MCPStreamableHTTPTool's request-header injection only ever runs for call_tool(). The initial connect() handshake (and any later reconnect) never carries the header_provider's headers, so a server that requires the header to authenticate the connect handshake itself gets a 401 before a single tool call is ever made.

Where this lives (checked against PyPI 1.0.1 and 1.15.0; 1.15.0 improves but does not fix this)

In agent-framework-core 1.0.1, get_mcp_client()'s httpx request hook reads a module-level contextvars.ContextVar (_mcp_call_headers) that is only ever .set() inside call_tool():

# get_mcp_client()
async def _inject_headers(request: Request) -> None:
    headers = _mcp_call_headers.get({})
    for key, value in headers.items():
        request.headers[key] = value
...

# call_tool()
if self._header_provider is not None:
    headers = self._header_provider(kwargs)
    token = _mcp_call_headers.set(headers)
    try:
        return await super().call_tool(tool_name, **kwargs)
    finally:
        _mcp_call_headers.reset(token)

connect() drives get_mcp_client() directly and never sets _mcp_call_headers, so the hook always sees {} during the initial handshake.

1.15.0 adds an "ambient request" fallback for when no call is in flight, which calls self._header_provider({}) (empty kwargs) on a request outside call_tool(). This helps a static header_provider (one that ignores its kwargs argument and always returns the same headers), but does nothing for a header_provider that legitimately depends on its kwargs argument (e.g. a provider that expects the caller to inject a per-turn secret via FunctionInvocationContext.kwargs, as our agent-runtime integration does): calling it with {} silently returns no headers, and the request still goes out unauthenticated. Worth noting explicitly in case this is read as already fixed.

Even a correctly-scoped fix at the call site would not be reliable, because connect() does not run inline: it dispatches through a lifecycle-owner task created via asyncio.create_task() (_ensure_lifecycle_owner, _run_lifecycle_owner). A contextvars.ContextVar set by the caller around connect() is only visible to that new task via the context snapshot taken at task-creation time; it is not visible to any later reconnect (a new owner task, created after the old one's finally clears _lifecycle_owner_task), and .reset() on the caller's own context cannot affect the task's copy anyway. So the real fix likely belongs inside get_mcp_client()/_connect_on_owner() itself, resolving and applying the header_provider's headers (with real kwargs where available, or a documented static-provider convention) at the point the client is actually built or the request is actually sent, not via a ContextVar set at an outer call site.

Reproduction

Live-reproduced against a real, deployed MCP server configured with header_provider-based bearer-token auth: a bare curl (no Authorization header) against the server's /mcp endpoint returns HTTP/2 401 with www-authenticate: Bearer error="invalid_token". connect() against that same server via MCPStreamableHTTPTool fails/hangs the same way, because it sends the same unauthenticated request. call_tool() against the same server, once connected another way, sends the header correctly.

Suggested fix

For header-provider-authenticated servers, apply the resolved headers to the client construction path itself (get_mcp_client()/_connect_on_owner()), not only around call_tool(). If a header_provider is present but the caller has not supplied kwargs (i.e. connect/discovery/reconnect), consider either:

• documenting that header_provider must tolerate being called with {} and return a best-effort static credential in that case, or
• providing a distinct, explicit hook for "headers needed at connect time" separate from the per-call kwargs-dependent provider, so a caller with genuinely per-call-dependent auth and genuinely connect-time-static auth are not forced into the same callable signature.

Workaround in use

We worked around this downstream (not in the vendored library) by giving header-provider-authenticated servers their own httpx.AsyncClient with the resolved headers baked in as client defaults (httpx.AsyncClient(headers=...)), rather than relying on header_provider/_mcp_call_headers for these servers at all. This sidesteps the ContextVar/task-lifecycle issue entirely, since default client headers apply to every request the client sends regardless of which task sends it.

Versions

• agent-framework-core / agent-framework-openai: reproduced on 1.0.1 (pinned in our project); confirmed not fixed (for kwargs-dependent providers) on 1.15.0 (latest at time of writing) by downloading the wheel and diffing _mcp.py.