Skip to content

Python: Honor Foundry Toolbox approval metadata - #10136

Merged
Eduard van Valkenburg (eavanvalkenburg) merged 5 commits into
microsoft:mainfrom
eavanvalkenburg:toolbox-approval-parsing-fix
Nov 6, 2026
Merged

Eduard van Valkenburg (eavanvalkenburg) merged 5 commits into
microsoft:mainfrom
eavanvalkenburg:toolbox-approval-parsing-fix

Conversation

@eavanvalkenburg

Copy link
Copy Markdown
Member

Motivation & Context

Foundry Toolbox discovery exposes approval configuration in each tool's _meta.tool_configuration.require_approval, but Python ignored that configuration. A live hosted Responses reproduction confirmed that a tool configured with require_approval: "always" executed without an mcp_approval_request. An explicit local approval policy paused the same tool correctly.

This fixes metadata-based approval selection before local tool execution, without interpreting Foundry-specific metadata for ordinary MCP servers.

Description & Review Guide

  • What are the major changes? Add synchronous callable policies to MCP approval_mode and compose the Foundry metadata parser through that option rather than an inheritance hook. Callables receive the discovered mcp.types.Tool or mcp.types.Prompt and must return "always_require" or "never_require" for every item, including their default decision. Invalid results, including None, fail discovery explicitly. FoundryToolbox.approval_mode is now an explicit, typed constructor parameter.
  • What is the impact of these changes? For Toolbox, omitted/None selects metadata-based approval. Explicit scalar, dictionary, or callable policies replace metadata handling entirely. Ordinary MCP None and static-policy behavior remain unchanged. Toolbox metadata supports "always", "never", and always/never filters using remote/original tool names and read_only. Filter criteria are combined, always wins overlaps, and unmatched filter-policy tools require approval. Missing metadata preserves the no-approval default; malformed metadata fails discovery instead of silently disabling approval. Callable policies refresh retained functions, with updates staged until the complete discovery snapshot validates. Failed refreshes preserve the previous snapshot and approval modes. Static policies preserve caller customizations on retained functions.
  • What do you want reviewers to focus on? Approval-policy precedence, exact remote-name matching, conservative filter resolution, and failure atomicity across paginated discovery and reconnects. Offline regressions exercise streaming/non-streaming approval and rejection, resume/replay, exactly-once execution, and the hosted Responses approval envelope.

Before, applications needed a blanket local override to guarantee an approval pause:

from agent_framework_foundry_hosting import FoundryToolbox
from azure.identity.aio import DefaultAzureCredential

credential = DefaultAzureCredential()
toolbox = FoundryToolbox(
    credential,
    url="https://toolbox.example/mcp",
    approval_mode="always_require",
)

After, the default honors each discovered tool's Foundry configuration:

from agent_framework_foundry_hosting import FoundryToolbox
from azure.identity.aio import DefaultAzureCredential

credential = DefaultAzureCredential()
toolbox = FoundryToolbox(
    credential,
    url="https://toolbox.example/mcp",
)

Use approval_mode="never_require" to disable local Toolbox approval explicitly. Connection and credential ownership, approval persistence, and the function-loop protocol are unchanged. Streaming and non-streaming execution use the same policy; both pause before tool execution when approval is required. The implementation and callable-based composition were reviewed and approved by the maintainer during the live-reproduction follow-up. The final patch was validated locally; the patched package has not been redeployed to the live reproduction.

Local validation: uv run --directory python poe test -P core passed (9,400 passed, 20 skipped, 2 xfailed); uv run --directory python poe test -P foundry_hosting passed (1,092 passed, 36 integration tests deselected). uv run --directory python poe syntax -P core, uv run --directory python poe syntax -P foundry_hosting, uv run --directory python poe pyright -P core, uv run --directory python poe pyright -P foundry_hosting, uv run --directory python poe test-typing -P core, and uv run --directory python poe test-typing -P foundry_hosting all passed. Both test-typing runs passed all five configured checkers. git diff --check passed.

Related Issue

No linked issue. This draft follows a reproduced bug and maintainer-reviewed implementation; issue linkage remains outstanding. No competing Python Toolbox approval-parsing PR was found.

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: Copilot assisted with investigation, implementation, regression tests, documentation, and this PR description.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI balanced review requested due to automatic review settings November 6, 2026 09:24
@agent-framework-automation agent-framework-automation Bot added documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python labels Nov 6, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

If prompt discovery fails during reconnect, shallow-copy rollback leaves retained tools with changed approval modes, potentially downgrading them from always requiring approval to never requiring it.

1 open issue
What changed in this PR

This PR makes Python Foundry Toolbox honor discovered approval metadata through a new callable MCP approval policy, while preserving ordinary MCP defaults and explicit caller overrides.

Changes:

  • Add callable approval policies and refresh approval modes on retained functions.
  • Parse Foundry approval metadata with conservative filter matching and validation.
  • Add offline approval-flow regressions and document policy precedence and behavior.
File Description
skills/​agent-framework-builder-python/​references/​hosting-and-deployment.md Document Toolbox approval behavior for consumers.
python/​packages/​foundry_hosting/​tests/​test_toolbox.py Cover policy precedence, refreshes, and approval flows.
python/​packages/​foundry_hosting/​tests/​foundry_hosting/​toolbox_approval_helpers.py Add offline MCP and model test helpers.
python/​packages/​foundry_hosting/​README.md Explain metadata filters and caller overrides.
python/​packages/​foundry_hosting/​pyproject.toml Enable pytest importlib mode.
python/​packages/​foundry_hosting/​agent_framework_foundry_hosting/​_toolbox.py Parse metadata and expose the approval policy parameter.
python/​packages/​core/​AGENTS.md Document callable policy and discovery semantics.
python/​packages/​core/​agent_framework/​_mcp.py Support callable policies and staged approval updates.

Review effort: Balanced

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): d1cf616874ea
Model: gpt-5.6-sol

Overview

The PR adds callable per-discovery approval policies in Core and uses them to map Foundry Toolbox metadata, with strict result validation, conservative unmatched-filter behavior, staged per-catalog refreshes, and execution-path regression coverage. Three residual risks remain: the Foundry package still supports Core versions that do not evaluate callable policies, a downstream MCP server can self-assert the read-only hint used to bypass approval, and tool approval updates can escape a failed combined tool/prompt reconnect.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
3 verified findings remained after source verification (2 high, 1 medium) across 2 files. Details are attached to the affected lines below.

Affected areas: python/packages/core/agent_framework/_mcp.py, python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 2

Result: No findings
Scope: 3 net-new commit(s): 21239115c4b5, 67e30274d06d, a7294b3cc8a0
Model: gpt-5.6-sol

Overview

The incremental change correctly makes failed MCP reconnects restore approval modes on retained function objects, including when prompt discovery fails after tool discovery has already refreshed policy. The restore runs even if connection cleanup raises, and the new regression covers policy changes in both directions plus a subsequent successful reconnect. No new publishable Critical, High, or Medium issue was established in this review range.

Reviewed the supplied incremental change set across correctness, security/reliability, architecture, and failure behavior.
No publishable findings remained after source verification for this scope.

Merged via the queue into microsoft:main with commit 9db7efb Nov 6, 2026
53 checks passed
@eavanvalkenburg
Eduard van Valkenburg (eavanvalkenburg) deleted the toolbox-approval-parsing-fix branch November 6, 2026 10:37

This branch was previously deployed

1 inactive deployment
github-app-auth — eaf7d1b2 Deployed Nov 6, 2026 by eavanvalkenburg via add_label #27248
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants