Python: [Bug]: header_provider not applied to MCP initialize() request

Description

header_provider on MCPStreamableHTTPTool only gets called from call_tool(). The initialize handshake sent during connect() goes out before any tool call happens, so it never gets headers — for MCP servers that require auth on initialize (Azure AI Search knowledge-base MCP endpoints, for example), this means an immediate 401 and the tool never connects.

Root cause is in _mcp.py — _header_provider is only invoked in MCPStreamableHTTPTool.call_tool(), and session.initialize() runs earlier in connect()/_connect_on_owner(), before any tool call could happen.

I haven't contributed to a large repo like this before, attempting to be as good of a steward as possible but may have missed guidance along the way. Happy to make any corrections.

Code Sample

\`\`\`python
def auth_headers(_kwargs: dict) -> dict[str, str]:
    print("called")  # never prints
    return {"Authorization": f"Bearer {get_token()}"}

tool = MCPStreamableHTTPTool(
    name="kb",
    url="https://<search-service>.search.windows.net/knowledgebases/<kb>/mcp?api-version=2025-11-01-Preview",
    header_provider=auth_headers,
)

agent = Agent(client=client, tools=[tool])
await agent.run("hello")  # 401 on initialize, header_provider never runs
\`\`\`

Error Messages / Stack Traces

`Error MCP server failed to initialize: Cancelled via cancel scope #####`

As seen when using a hosted Foundry agent to connect to an Azure AI Search Knowledgebase

Package Versions

agent-framework-core 1.1.0

Python Version

3.1.2

Additional Context

No response

🤖 Automated triage reproduction notes (agent-authored — trust but verify)

Agent analysis

Repro: MCPStreamableHTTPTool in packages/core/agent_framework/_mcp.py::call_tool (line 3104) is the only location that invokes _header_provider and sets the _mcp_call_headers ContextVar; _connect_on_owner (line 1263) calls session.initialize() (line 1343) before any call_tool, so the httpx event hook _inject_headers (line 3071) reads an empty ContextVar and injects no headers. Minimal repro: construct MCPStreamableHTTPTool with a header_provider, call get_mcp_client(), then fire an HTTP request through tool._httpx_client — the Authorization header is absent.

Failing test: python/packages/core/tests/core/test_mcp_header_provider_initialize_bug.py
Files examined: python/packages/core/agent_framework/_mcp.py, python/packages/core/tests/core/test_mcp.py, python/packages/core/pyproject.toml
Tests run: tests/core/test_mcp_header_provider_initialize_bug.py::test_header_provider_not_called_during_initialize, tests/core/test_mcp_header_provider_initialize_bug.py::test_header_provider_headers_missing_from_initialize_request, tests/core/test_mcp_header_provider_initialize_bug.py::test_header_provider_only_called_in_call_tool_not_connect
Reported version: 1.1.0
Current version: 1.11.0

Eric Frankenberger (@efranken)

Hello everyone,

I did a deep dive into the codebase to trace the root cause and evaluate the safest, most robust fix for this issue. Below is a detailed breakdown of the findings, the design alternatives I evaluated, and the local verification, structured in accordance with the project's contributing guidelines.

1. High-Level Problem & Root Cause

In agent_framework, outgoing HTTP requests for MCPStreamableHTTPTool pass through the _inject_headers request event hook (registered on the httpx.AsyncClient). This hook is responsible for fetching headers from the _mcp_call_headers ContextVar and injecting them into the request.

However, _mcp_call_headers is only set within MCPStreamableHTTPTool.call_tool(). Consequently, any HTTP requests sent outside of a tool call execution context are missing their authentication headers.

This impacts the following lifecycle phases:

Phase 1: Eager Connection (connect / initialize)
When the tool context is entered (__aenter__), it triggers connect(), which starts the session.initialize() handshake. Since this runs before any tool calls, the hook injects no headers, leading to a 401 Unauthorized response on authenticated endpoints.
Phase 2: Lifecycle Updates (load_tools / load_prompts / send_ping)
Any dynamic updates or background pings executed after connection also run outside call_tool(), so they fail with 401 Unauthorized as well.
Phase 3: Runtime Execution (call_tool)
Tool calls succeed because they wrap the execution context and populate the ContextVar.

2. Proposed Design & Alternatives

Recommended Approach: Add an Ambient Fallback in the Subclass Hook

Update the _inject_headers request event hook in MCPStreamableHTTPTool.get_mcp_client() to support an ambient fallback:

headers = _mcp_call_headers.get(None)
if headers is None:
    headers = self._header_provider({})

Benefits

Encapsulation: Keeps all HTTP-specific header logic contained within MCPStreamableHTTPTool.
Complete Coverage: Every HTTP request—including handshake, discovery, background pings, and tool calls—automatically receives the appropriate headers.
Dynamic Refresh: Calls _header_provider({}) on demand for ambient requests, allowing dynamically refreshed credentials (for example, OAuth or managed identity tokens).
Lazy Connection Support: If the tool connects lazily on the first tool call, the ContextVar is already populated, so the fallback is naturally bypassed.

Alternative 1: Wrap `connect_on_owner` in the Base Class

Design: Wrap the connection block in _connect_on_owner (defined in _mcp.py) to temporarily populate the ContextVar using getattr(self, "_header_provider").

Drawbacks

Breaks Encapsulation: _connect_on_owner belongs to the shared MCPTool base class (used by stdio and WebSocket implementations). Accessing _header_provider, which is HTTP-specific, leaks subclass details into the base class.
Incomplete Coverage: Dynamic refreshes and background pings executed after connect() completes would still run without authentication.

Alternative 2: Configure Default Client Headers During Construction

Design: Call _header_provider({}) during client creation and set the returned headers as the default headers on the httpx.AsyncClient.

Drawbacks

No Dynamic Token Refresh: The headers become fixed at client creation time, so expiring credentials cannot be refreshed automatically.
Added Complexity: Call-specific headers would need to override the default client headers, making the event hook more complicated than necessary.

Local Verification

I verified the recommended approach locally by:

Adding a new unit test, test_mcp_streamable_http_tool_header_provider_injects_on_connect, in python/packages/core/tests/core/test_mcp.py to verify that headers are injected during the connection/handshake phase.
Running the full MCP test suite (pytest packages/core/tests/core/test_mcp.py), with all 272 tests passing.
Running the formatting and lint checks (uv run poe syntax -P core), which completed successfully.

Eric Frankenberger (@efranken) and Giles Odigwe (@giles17) Evan Mattson (@moonbox3) : I'd love to hear your thoughts on this approach.

Appreciate your review. My PR is locally scoped to my issue, whereas your proposal is global and catches more failures than I experienced.

I believe the proposal will encounter a key error when running mcp_api_key_auth.py. it indexes the mcp_api_key which is only present after agent.run, which would happen after connect().

Proposal is to either modify the sample (potentially breaking existing solutions built using the sample) or add a try/except:

                   headers = _mcp_call_headers.get(None)
                   if headers is None:
                       try:
                           headers = self._header_provider({})
                       except Exception:
                           logger.warning(
                               "header_provider raised for MCP server %r on an ambient request; proceeding without headers.",
                               self.name,
                               exc_info=True,
                           )
                           headers = {}

I can modify or happy to defer to another PR.

Eric Frankenberger (@efranken)

Independent reproduction, with a couple of details that I think are still missing from the thread.

Still present on 1.12.1. The report was against 1.1.0 and the triage run against 1.11.0, so this confirms nothing has landed in between.

Not Azure-specific. Every repro here so far is Azure AI Search or Foundry. Mine is an unrelated third-party public MCP server (Unstructured Transform, streamable HTTP, bearer auth on the handshake), so the impact is any MCP server that authenticates initialize, not just Microsoft-hosted endpoints.

Wire-level evidence. Attaching httpx event hooks to the client and checking whether the header reached the request:

('REQ', 'POST', 'https://mcp.transform.unstructured.io', Authorization_present=False)
('RES', 401, 'https://mcp.transform.unstructured.io', Authorization_present=False)

One request, header absent, 401. header_provider is never consulted.

Same server, same key, only difference being the auth is on the client instead:

client = httpx.AsyncClient(headers={"Authorization": f"Bearer {KEY}"})
tool = MCPStreamableHTTPTool(name="...", url=URL, http_client=client)
async with tool:
    ...  # connects, lists all 7 tools

So the transport is fine, it is purely that the handshake never sees the provider's headers.

One diagnosability note. The surfaced error is ToolException: MCP server failed to initialize: Cancelled via cancel scope 109e0c730, which is what Eric Frankenberger (@efranken) saw too. The underlying 401 is not visible anywhere in that message, so from the outside an auth failure is indistinguishable from a network or timeout problem. Even with the header behaviour left as-is, surfacing the HTTP status from the failed handshake would save people a lot of time.

On the sample question above: worth noting that until this is resolved, python/samples/02-agents/mcp/mcp_api_key_auth.py cannot work against a server that authenticates initialize, and its docstring currently recommends header_provider over "being baked into a shared httpx.AsyncClient", which is the pattern that does work today. If the fix ends up being scoped rather than global, it may be worth a note in that sample either way.

Happy to re-run the repro against any candidate fix.

Navigation Menu

Repository navigation

Python: [Bug]: header_provider not applied to MCP initialize() request #7079

Description

Code Sample

Error Messages / Stack Traces

Package Versions

Python Version

Additional Context

1. High-Level Problem & Root Cause

2. Proposed Design & Alternatives

Recommended Approach: Add an Ambient Fallback in the Subclass Hook

Alternative 1: Wrap `connect_on_owner` in the Base Class

Alternative 2: Configure Default Client Headers During Construction

Local Verification

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Python: [Bug]: header_provider not applied to MCP initialize() request #7079

Description

Description

Code Sample

Error Messages / Stack Traces

Package Versions

Python Version

Additional Context

Activity

github-actions commented on Jul 12, 2026

abhishekabramaina commented on Jul 13, 2026

1. High-Level Problem & Root Cause

2. Proposed Design & Alternatives

Recommended Approach: Add an Ambient Fallback in the Subclass Hook

Alternative 1: Wrap connect_on_owner in the Base Class

Alternative 2: Configure Default Client Headers During Construction

Local Verification

efranken commented on Jul 13, 2026

simoncoombes commented on Jul 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Alternative 1: Wrap `connect_on_owner` in the Base Class