Repository navigation
Description
Description
In the Python core, a tool configured with approval_mode="always_require" can be silently bypassed (executed or returned as user input without surfacing an approval request) when another call precedes it in the same model tool-call batch. The approval outcome depends on the position of the call within the batch rather than on the call itself.
Root cause
_try_execute_function_call_groups in python/packages/core/agent_framework/_tools.py classifies the batch with a loop that stops at the first matching call:
for function_call in actionable_calls:
function_name = function_call.name
if function_name in approval_tool_names:
requires_approval = True
break
if function_name in declaration_only_tool_names or function_name in additional_tool_names:
has_declaration_only_call = True
break
if config.get("terminate_on_unknown_calls", False) and function_name not in tool_map:
raise KeyError(f'Error: Requested function "{function_name}" not found.')Because each branch breaks (or raises) at the first hit, classification travels with the call's position, not with the call:
- A declaration-only call appearing before an
always_requirecall makes the whole batch classified as declaration-only user input, so the approval gate is never surfaced. - With
terminate_on_unknown_calls=True, an unknown call appearing before analways_requirecall raisesKeyErrorfirst, so the higher-priority approval pause is never reached.
Reproduction
Model returns one batch with two calls where a declaration-only (or unknown) call is ordered first and an always_require tool is second. Expected: an approval request is surfaced for the always_require tool. Actual: no approval request; the batch is returned as user input (declaration-first) or terminated with KeyError (unknown-first).
Expected invariant
Approval enforcement must be independent of call order within a batch. Classification should scan the whole batch and apply a fixed priority: approval pause > declaration-only user-input > unknown-call termination.
Scope
python/packages/core/agent_framework/_tools.py batch classification in _try_execute_function_call_groups. Verified present on current main.
Note: this is distinct from #6385 (which concerned the opposite direction, never_require tools being over-approved) and from #7881 (a concurrency-groups feature that retains the current order-dependent classification).
I'm working on a fix with regression tests and will open a PR shortly with Fixes #NNNN.
Metadata
Metadata
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
he-yufeng commentedon Sep 4, 2026
Confirmed on current main, and there is a nice smoking gun: the comment right above the loop says "A user-input pause takes precedence over unknown-call termination in mixed batches", but the break-on-first-hit loop cannot honor that invariant unless the pause-worthy call happens to come first. All three orderings are broken the way you describe: declaration-only before always_require masks the approval, unknown before always_require (or before declaration-only) raises KeyError before the pause is ever considered.
Fix I would ship: scan the whole batch with explicit precedence instead of first-hit-wins. Collect requires_approval / has_declaration_only_call across all actionable calls (approval short-circuits only upward), and evaluate terminate_on_unknown_calls only after, when neither pause classification fired. That makes the code match its own comment, and single-classification batches behave exactly as before. Tests: the two masked orderings from your report, the unknown+terminate mixed batch, and the pure unknown+terminate case still raising. Happy to open the issue-linked draft.
he-yufeng commentedon Sep 4, 2026
One process note before I open anything: the function-calling-loop spec (docs/specs/004) asks external contributors to check with the core team before implementing in this area, so I am holding the draft until someone from the team confirms the intended precedence and the affected scenario-matrix rows.
The fix and regression tests are ready locally and follow the shape above: whole-batch classification with explicit precedence (approval > declaration-only pause > unknown-call termination), unknowns fatal only when neither pause fired. Tests pin the two masked orderings, the unknown+terminate mixed batch, and that a lone unknown call still raises. Happy to open the issue-linked draft the moment the team gives direction.
CorgiBoyG commentedon Sep 5, 2026
Thanks for calling out the ownership guidance. I had already opened #8080 with the reproduction and regression coverage, but the core team should confirm the intended precedence, affected scenario-matrix rows, and validation scope before this moves forward. I've converted the PR to draft and will hold it there. Happy to adjust or step back based on the team's direction; this should also avoid duplicate implementations.
he-yufeng commentedon Sep 5, 2026
Status update for the core team's precedence call: I verified #8080 locally against current main (red on main's
_tools.pyfor the order-independence and approval-resume tests, green on the branch, no regressions in the full invocation-logic suite) and left the details there. My own draft, which keeps the pause-first reading of the old comment, stays on ice. Whichever precedence the team confirms, the fix shape is ready in one of the two drafts.github-actions commentedon Sep 7, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
Repro:
python/packages/core/agent_framework/_tools.py::_try_execute_function_call_groupsat lines 178-1804 fails when a declaration-only or unknown call precedes analways_requirecall in one batch. The declaration-first case emits no approval request, while unknown-first raisesKeyError; reversing either ordering surfaces approval. Minimal repro: return those two calls from the mock chat client, enableterminate_on_unknown_callsfor the unknown case, and assert an approval request for the second call.python/packages/core/tests/core/test_function_invocation_logic.py::test_always_require_approval_is_independent_of_batch_orderShivani767 commentedon Sep 9, 2026
I can take this up once the intended precedence is confirmed by the core team.
From the current implementation, I think the safest fix is to make the batch classification independent of call order rather than relying on the first matching call. I would scan the actionable calls first and track the classification flags, then apply the precedence explicitly:
always_require approval > declaration-only user input > unknown-call terminationIn particular, an unknown call should not raise immediately if the same batch contains a tool that requires approval, since we need to surface the approval request first.
I’d add regression coverage for:
terminate_on_unknown_calls=Trueterminate_on_unknown_calls=TrueI’m happy to implement this and take ownership of the fix. If the core team is okay with this approach, could you assign the issue to me? I can work from the existing draft/coordinate with the current PR rather than creating a duplicate implementation.
Shivani767 commentedon Sep 9, 2026
Implemented and pushed in 9899e37.
Fix
The batch classifier now scans all actionable function calls before selecting a control-flow path. This makes approval enforcement independent of the call order within a model tool-call batch.
The resulting precedence is:
always_requireapproval pauseterminate_on_unknown_calls=TrueRegression coverage
always_requirecallalways_requirecall with unknown-call termination enabledValidation
I kept this on a separate branch and did not open a duplicate PR because #8080 is already tracking the issue.
CorgiBoyG commentedon Sep 10, 2026
Thanks for keeping this on a branch and not opening a duplicate PR. A few coordination points are worth keeping explicit before
9899e37is treated as the implementation direction:approval > declaration-only > unknown), while Python: fix order-dependent tool-approval bypass in mixed batches #8080 currently implements fail-closed unknown-call precedence. That contract is still awaiting the core team's decision.6a3633e54(Aug 17), rather than currentmain.Since #8080 is already linked and I have local follow-ups for those paths, let's avoid opening another PR until the core team confirms the precedence and scenario-matrix rows. Once that direction is clear, I'm happy to reconcile useful regression cases into the existing PR and coordinate review.
ozereray commentedon Sep 10, 2026
AEGISORA-W29-MAF
The batch-order approval bug looks like a strong candidate for an execution-boundary invariant.
I would express the expected behavior as:
For a batch of tool calls, approval requirements must be determined from the complete batch, not from whichever call happens to appear first.
Adversarial fixture:
call 1 = declaration-only or unknown
call 2 = always_require tool
Expected:
approval is surfaced for call 2
Security invariant:
no approval-required action may execute merely because an earlier call changed batch classification.
The same fixture can also measure the stronger property:
before = N downstream tool executions
after unauthorized approval bypass attempt = N
That makes the failure observable at the execution boundary rather than only through control-flow state.
We are testing this class of invariant in Aegisora's public runtime-security challenge:
aegisora-ai/aegisora#28
Shivani767 commentedon Sep 10, 2026
Thanks for the clarification, RongJie G (@CorgiBoyG) That makes sense.
I’ll hold off on opening a separate PR and won’t treat
9899e37as the final implementation direction until the core team confirms the intended precedence and scenario-matrix coverage.Once the direction is confirmed, I’m happy to rebase my work onto current
main, reconcile the additional approval-resume/declaration-only lifecycle cases, and contribute the relevant regression tests to #8080 rather than creating a parallel implementation.I’ll keep the branch available so the work can be reused if the pause-first precedence is confirmed.
Shivani767 commentedon Sep 10, 2026
Thanks, Eray Özer (@ozereray) ! I agree that the execution boundary is the right place to validate the invariant.
For this issue, I’d keep the primary fix focused on the in-framework guarantee: an
always_requiretool must not be executable or returned without an approval request simply because another call appeared earlier in the same batch.An external artifact could be useful as additional evidence, but I think the important contract here is that approval classification is derived from the complete batch and remains independent of call ordering. Once the core team confirms the precedence and scenario matrix, the regression tests in #8080 can make that invariant concrete at the framework level.
MkaliezZ commentedon Sep 10, 2026
Hi Corgi,
Your investigation into approval depending on call position rather than the specific call itself raised an interesting question.
Beyond fixing the ordering issue, there is also a broader challenge: after an approval decision happens, how can we clearly observe that the specific call was the one actually allowed or blocked?
I have been testing a small execution-boundary experiment around this idea — keeping the scope to one local tool call and looking at what can be verified before and after entry.
Would be interested in your thoughts on whether this kind of evidence is useful when reasoning about approval semantics.
ozereray commentedon Sep 10, 2026
MkaliezZ commentedon Sep 14, 2026
fix: enforce tool approval regardless of call order in mixed batches
ozereray commentedon Sep 15, 2026
fix: enforce tool approval regardless of call order in mixed batches
eavanvalkenburg commentedon Sep 16, 2026
Thanks everyone for the investigation and test suggestions. I implemented a fix in #8428.
The function loop now:
ToolApprovalMiddlewarecontinues to support standing and automatic approval policies. It preserves Host-owned requests in the batch and does not execute cached calls until the required responses are available.The regression tests cover:
Incomplete mixed responses fail closed and can be retried with the complete response set. The function-calling-loop specification has been updated with this behavior.
MkaliezZ commentedon Sep 16, 2026