Repository navigation
.NET: .NET/Python: design per-tool concurrency controls #7914
Description
Per-tool concurrency groups were proposed in #7881 so related stateful tools can execute sequentially while unrelated tools remain concurrent. We should defer that portion of #7881 and design it independently across SDKs rather than introducing a Python-only API before its semantics and integration points are settled.
Questions the design should answer include:
- Should per-tool concurrency controls be introduced in both .NET and Python at the same time, with aligned naming and semantics?
- How should developers configure discovered MCP tools: one group per MCP server, mappings of tool names to groups, post-discovery mutation, or MCP metadata/annotations?
- How should locally executed MCP tools differ from provider-hosted MCP tools, which bypass the local function-invocation scheduler?
- Are groups scoped only to calls in one model-response batch, or should they coordinate across concurrent runs, agents, sessions, or shared tool instances?
- What are the required middleware semantics when a call fails closed or requests termination? No later queued side-effecting call should start after the batch has been stopped.
- How should approval pauses and replay preserve original model call order?
- What cancellation guarantees can be made for async functions versus synchronous functions already running in worker threads?
- How should user-defined group identifiers be represented without colliding with scheduler-internal keys?
- How should dynamically added tools, MCP reloads, and progressive disclosure inherit or retain concurrency policy?
- What result-ordering and observability guarantees should be shared across SDKs?
This issue is specifically about selective, tool-level controls. Batch-wide execution policy should remain separate: Python can add allow_concurrent_invocation to FunctionInvocationConfiguration, mirroring .NET, while retaining Python's existing parallel default.
Roger Barreto (@rogerbarreto), could you advise whether this should be designed and implemented for .NET and Python simultaneously?
Activity
[-].NET/Python: design per-tool concurrency controls[/-][+]Python: .NET/Python: design per-tool concurrency controls[/+][-]Python: .NET/Python: design per-tool concurrency controls[/-][+].NET: .NET/Python: design per-tool concurrency controls[/+]JHf0912 commentedon Sep 4, 2026
Hi! I'd like to contribute to the design discussion here. My initial outline:
1. Configuration: a
tool_groupsmapping on tool registration, plus a convention for discovered MCP tools — one group per server by default, overridable via MCP tool annotations (reusing the annotations channel avoids inventing a new metadata protocol that non-MAF hosts cannot share).2. Scope: groups coordinate within one model-response batch first, alongside the existing
_FunctionExecutionBatchscheduler; cross-run coordination is deferred to a second phase since it interacts with approval replay and checkpointing.3. Scheduler semantics: batches execute group-by-group in submission order — groups serialized relative to each other, tools within a group keep current concurrency. Fail-closed: once termination is requested, no queued call in the batch starts.
4. Approval interplay: pauses persist pending requests to the session; group execution order must be part of that persisted state so replay preserves original ordering.
Two questions before I write a full design doc: should this be submitted as an ADR under
docs/decisions/, and is simultaneous .NET/Python implementation the expectation for the first PR?