Skip to content

Commit af5572f

Browse files
Python: Preserve grouped approval pauses in AG-UI
Keep approval-time follow-up requests grouped through the AG-UI adapter and count every executed result group against the function-call budget in both response modes. Co-authored-by: Copilot <[email protected]> Copilot-Session: 1ee1d250-d1e2-4c6f-8c36-aae0d94fe7a1
1 parent 6476bf6 commit af5572f

7 files changed

Lines changed: 260 additions & 59 deletions

File tree

‎docs/specs/004-python-function-calling-loop.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -391,12 +391,12 @@ that manually replay messages own the equivalent rule: do not resend an approval
391391
| Initial approval request | Assistant response contains the original call and approval request; tool does not execute. | `test_approval_requests_in_assistant_message`, `test_streaming_approval_request_generated`, `test_streaming_approval_requests_in_assistant_message` |
392392
| Approved non-streaming resume | Result precedes final text; tool executes once; inputs remain unchanged. | `packages/core/tests/core/test_harness_tool_approval.py::test_approval_resume_returns_result_without_mutating_inputs[non-streaming-approved]` |
393393
| Rejected non-streaming resume | Rejection result precedes final text; tool executes zero times; inputs remain unchanged. | `test_approval_resume_returns_result_without_mutating_inputs[non-streaming-rejected]` |
394-
| Approved streaming resume | Result update precedes final text and final response matches non-streaming shape. | `test_approval_resume_returns_result_without_mutating_inputs[streaming-approved]` |
395-
| Rejected streaming resume | Rejection result update precedes final text and tool executes zero times. | `test_approval_resume_returns_result_without_mutating_inputs[streaming-rejected]` |
394+
| Approved streaming resume | Result update precedes final text and final response matches non-streaming shape. | `test_approval_resume_returns_result_without_mutating_inputs[streaming-approved]`, `test_streaming_approval_resume_yields_terminal_result_before_model_text[approved]` |
395+
| Rejected streaming resume | Rejection result update precedes final text and tool executes zero times. | `test_approval_resume_returns_result_without_mutating_inputs[streaming-rejected]`, `test_streaming_approval_resume_yields_terminal_result_before_model_text[rejected]` |
396396
| Mixed approved/rejected batch | Every call gets one correctly correlated terminal result. | `packages/core/tests/core/test_function_invocation_logic.py::test_rejected_approval` |
397397
| Persisted approval replay | Resume executes with the prior call available. | `test_persisted_approval_messages_replay_correctly` |
398398
| Hosted approval pass-through | Hosted requests/responses are not processed as local calls. | `test_hosted_tool_approval_response`, `test_hosted_mcp_approval_response_passthrough`, `test_mixed_local_and_hosted_approval_flow` |
399-
| Approval-time user input | Every user-input request from one approved execution returns in order with assistant role and no extra model call. | `packages/core/tests/core/test_harness_tool_approval.py::test_approval_resume_returns_all_user_input_requests_without_another_model_call` |
399+
| Approval-time user input | Every user-input request from one approved execution returns in order with assistant role and no extra model call; the execution consumes one call-budget unit. | `packages/core/tests/core/test_harness_tool_approval.py::test_approval_resume_returns_all_user_input_requests_without_another_model_call`, `packages/core/tests/core/test_function_invocation_logic.py::test_approval_resume_user_input_counts_toward_function_call_budget` |
400400
| Approval-time middleware termination | Terminal result returns with no extra model call in either response mode. | `packages/core/tests/core/test_function_invocation_logic.py::test_approval_resume_honors_middleware_termination` |
401401
| Approval re-entry after iteration budget | Pending approved calls resolve once even when prior model calls consumed `max_iterations`. | `packages/core/tests/core/test_harness_tool_approval.py::test_auto_approval_resolves_after_iteration_budget_is_exhausted` |
402402
| Approval resume with reasoning | Model-bound resume history retains reasoning before the call and terminal result in both modes. | `packages/core/tests/core/test_harness_tool_approval.py::test_approval_resume_replays_reasoning_with_function_call_group` |
@@ -411,6 +411,7 @@ that manually replay messages own the equivalent rule: do not resend an approval
411411
| Historical resolved response plus new round | The old response is removed from normalized input and is not converted into a rejection result. | `test_replace_approval_contents_with_results_ignores_already_resolved_response` |
412412
| Multiple reused-id rounds | Approved and rejected rounds retain separate call/result occurrences. | `test_replace_approval_contents_with_results_correlates_reused_call_id_occurrences` |
413413
| Multi-content result with reused id | Every content produced by one execution stays with that approval occurrence and cannot bleed into the next reused-id round. | `test_replace_approval_contents_with_results_keeps_multi_content_group_with_reused_call_id` |
414+
| Follow-up request closes one occurrence | A user-input follow-up consumes only the preceding approval authority and leaves a later reused-id response pending. | `test_collect_approval_responses_consumes_matching_follow_up_request_occurrence` |
414415
| Reused-id placeholders | Placeholder results consume approved results by occurrence. | `test_replace_approval_contents_with_results_correlates_reused_call_id_placeholders` |
415416
| Rejected placeholder | Rejection replaces the pending placeholder instead of adding a second result. | `test_replace_approval_contents_with_results_replaces_rejected_placeholder` |
416417
| Results reordered with placeholders | Results still match the correct call ids. | `test_replace_approval_contents_with_results_uses_result_call_ids_for_placeholders` |
@@ -431,7 +432,7 @@ that manually replay messages own the equivalent rule: do not resend an approval
431432
| Queued approvals | One unresolved approval is surfaced per run without premature execution. | `test_tool_approval_middleware_queues_multiple_approval_requests`, `test_tool_approval_middleware_queues_streamed_approval_requests` |
432433
| Middleware state plus hidden core state | State saves do not discard hidden mixed-batch calls. | `test_tool_approval_middleware_preserves_hidden_mixed_batch_requests` |
433434
| Auto-approval callback | Callback receives the original function call and executes the approved set once. | `test_tool_approval_middleware_auto_approval_rule_receives_function_call` |
434-
| Shared call budget | Auto-approved re-entry does not reset `max_function_calls`. | `test_tool_approval_middleware_auto_approved_loops_share_function_call_budget` |
435+
| Shared call budget | Auto-approved re-entry does not reset `max_function_calls`, and every executed approval group counts even when it pauses for input. | `test_tool_approval_middleware_auto_approved_loops_share_function_call_budget`, `test_approval_resume_user_input_counts_toward_function_call_budget` |
435436
| Standing tool rule | Tool-level approval applies only to later matching tools. | `test_tool_approval_middleware_always_approve_tool_rule` |
436437
| Hosted server boundary | Standing approval does not cross `server_label`. | `test_tool_approval_middleware_standing_rules_include_hosted_server_boundary` |
437438
| Argument-scoped rule | Exact arguments are required; empty arguments are not tool-wide. | `test_tool_approval_middleware_always_approve_tool_with_arguments_rule`, `test_tool_approval_middleware_empty_arguments_rule_is_not_tool_wide` |
@@ -445,11 +446,11 @@ that manually replay messages own the equivalent rule: do not resend an approval
445446
| Approved tool exception | Generic and detailed error modes preserve one result and one execution. | `test_approved_function_call_with_error_without_detailed_errors`, `test_approved_function_call_with_error_with_detailed_errors` |
446447
| Approved validation error | Validation failure returns one result without invoking the function body. | `test_approved_function_call_with_validation_error` |
447448
| Approved success | Successful approved execution returns one result. | `test_approved_function_call_successful_execution` |
448-
| Consecutive error cap | Error threshold stops repeated failures and submits collected results. | `test_function_invocation_config_max_consecutive_errors`, `test_streaming_function_invocation_config_max_consecutive_errors`, `test_approval_resume_error_limit_forces_final_no_tool_response` |
449+
| Consecutive error cap | Error threshold stops repeated failures, submits collected results, and makes only the required final no-tool model call. | `test_function_invocation_config_max_consecutive_errors`, `test_streaming_function_invocation_config_max_consecutive_errors`, `test_approval_resume_error_limit_forces_final_no_tool_response` |
449450
| Unknown call handling | Configured false returns an error result; configured true raises. | `test_function_invocation_config_terminate_on_unknown_calls_false`, `test_function_invocation_config_terminate_on_unknown_calls_true`, streaming equivalents |
450451
| Middleware termination | Normal non-approval loop stops without a second model call. | `test_terminate_loop_single_function_call`, `test_terminate_loop_multiple_function_calls_one_terminates`, `test_terminate_loop_streaming_single_function_call` |
451452
| Maximum iterations | No orphan calls; a final no-tool response or deterministic fallback is returned. | `test_max_iterations_limit`, `test_max_iterations_no_orphaned_function_calls`, `test_max_iterations_makes_final_toolchoice_none_call`, `test_max_iterations_blank_final_fallback_synthesizes_message`, streaming equivalents |
452-
| Maximum function calls | Parallel overshoot is bounded after the batch; blank final responses get fallback content. | `test_max_function_calls_limits_parallel_invocations`, `test_max_function_calls_single_calls_per_iteration`, `test_max_function_calls_blank_final_fallback_synthesizes_message`, streaming equivalent |
453+
| Maximum function calls | Parallel overshoot is bounded after the batch; every executed result group counts even without a `function_result`; blank final responses get fallback content. | `test_max_function_calls_limits_parallel_invocations`, `test_max_function_calls_single_calls_per_iteration`, `test_user_input_request_multiple_contents_propagate`, `test_approval_resume_user_input_counts_toward_function_call_budget`, `test_max_function_calls_blank_final_fallback_synthesizes_message`, streaming equivalent |
453454
| Provider tool content after an active limit | A call or approval request returned despite `tool_choice="none"` is removed in both response modes while metadata-only streaming updates remain visible. | `test_function_invocation_limit_drops_unexecutable_tool_content`, `test_streaming_function_invocation_limit_drops_unexecutable_tool_content`, `test_streaming_function_invocation_limit_preserves_metadata_after_tool_content_is_dropped` |
454455
| Conversation continuation | Conversation id updates between iterations and is cleared on stop where required. | `test_conversation_id_updated_in_options_between_tool_iterations`, `test_function_invocation_stop_clears_conversation_id_non_stream`, `test_streaming_function_invocation_stop_clears_conversation_id` |
455456

@@ -468,6 +469,7 @@ that manually replay messages own the equivalent rule: do not resend an approval
468469
| Chat Completions approval wrappers | Framework approval wrappers are not sent as chat messages. | `packages/openai/tests/openai/test_openai_chat_completion_client.py` approval serialization tests |
469470
| AG-UI approval result event | Approved result emits once with content and persists in snapshot. | `packages/ag-ui/tests/ag_ui/test_approval_result_event.py::test_approval_resume_emits_tool_call_result`, `test_approval_resume_result_has_content`, `test_approval_resume_snapshot_replaces_approval_payload_with_tool_result`, `test_approval_resume_zero_updates_emits_tool_result` |
470471
| AG-UI rejection/mixed decision | Transport emits only the events defined for approved and rejected calls without duplicates. | `test_rejection_does_not_emit_tool_call_result`, `test_mixed_approve_reject_emits_only_approved_tool_result`, `test_resolve_approval_responses_returns_only_approved` |
472+
| AG-UI approval-time follow-up | The full grouped user-input pause remains in message history and emits no synthetic `TOOL_CALL_RESULT`. | `test_resolve_approval_responses_preserves_follow_up_user_input_group` |
471473
| AG-UI no-approval path | Ordinary tool results do not gain an extra approval result event. | `test_no_approval_no_extra_tool_result` |
472474
| AG-UI `confirm_changes` snapshot | The synthetic confirmation result is replaced only by the result for its original function call; rejection and missing-result fallbacks contain no approval payload. | `packages/ag-ui/tests/ag_ui/test_confirm_changes_snapshot.py` |
473475
| Compaction pair integrity | Adjacent and non-adjacent function call/result groups remain atomic without pairing ambiguous or out-of-order ids. | `packages/core/tests/core/test_compaction.py::test_group_annotations_keep_tool_call_and_tool_result_atomic`, `test_group_annotations_include_reasoning_in_tool_call_group`, `test_group_annotations_pair_nonadjacent_function_result_by_call_id`, `test_group_annotations_pair_multiple_nonadjacent_results_with_declaration`, `test_group_annotations_do_not_pair_ambiguous_duplicate_call_ids` |

‎python/packages/ag-ui/AGENTS.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ AG-UI protocol integration for building agent UIs with the AG-UI standard.
2929
- Multimodal user inputs support both legacy (`text`, `binary`) and draft-style (`image`, `audio`, `video`, `document`) shapes.
3030
- Interrupted runs complete with `RUN_FINISHED.outcome.type == "interrupt"` and canonical `outcome.interrupts`; do not document or add new flows that depend on the legacy top-level `RUN_FINISHED.interrupt` field.
3131
- `Interrupt` and `ResumeEntry` come from the `ag-ui-protocol` package (`ag_ui.core`), not from an Agent Framework-specific interrupt model.
32+
- Approval-time execution preserves each call's complete result group. Follow-up user-input requests remain in the
33+
resumed messages, while `TOOL_CALL_RESULT` events are emitted only for terminal `function_result` contents.
3234
- SSE keepalive is endpoint-owned transport behavior configured through
3335
`add_agent_framework_fastapi_endpoint(keepalive_seconds=...)`. It emits SSE comments only; do not add `PING`,
3436
`HEARTBEAT`, or `KEEPALIVE` AG-UI events, and do not add runner-level keepalive settings.

‎python/packages/ag-ui/agent_framework_ag_ui/_agent_run.py‎

Lines changed: 17 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@
4343
_get_tool_map, # type: ignore
4444
_replace_approval_contents_with_results, # type: ignore
4545
_TOOL_APPROVAL_STATE_KEY, # type: ignore
46-
_try_execute_function_calls, # type: ignore
46+
_try_execute_function_call_groups, # type: ignore
4747
normalize_function_invocation_configuration,
4848
)
4949
from agent_framework._types import ResponseStream
@@ -1307,19 +1307,15 @@ async def _resolve_approval_responses(
13071307
approved_responses = validated
13081308
rejected_responses = validated_rejected
13091309

1310-
approved_function_results: list[Any] = []
1310+
approved_function_result_groups: list[list[Content]] = []
13111311

13121312
# Partition approved responses into static (execute now) and deferred (execute during run)
13131313
tool_map = _get_tool_map(tools) if tools else {}
1314-
static_approved = []
1315-
deferred_approval_ids = set()
1314+
static_approved: list[Content] = []
13161315

13171316
for approval in approved_responses:
13181317
tool_name = approval.function_call.name if approval.function_call else None
1319-
if tool_name not in tool_map:
1320-
# Provider-injected tool — defer to harness for execution during run
1321-
deferred_approval_ids.add(approval.id or "")
1322-
else:
1318+
if tool_name in tool_map:
13231319
static_approved.append(approval)
13241320

13251321
# Execute only statically-available approved tool calls
@@ -1333,36 +1329,31 @@ async def _resolve_approval_responses(
13331329
# Filter out AG-UI-specific kwargs that should not be passed to tool execution
13341330
tool_kwargs = {k: v for k, v in run_kwargs.items() if k != "options"}
13351331
try:
1336-
results, _ = await _try_execute_function_calls(
1332+
approved_function_result_groups, _ = await _try_execute_function_call_groups(
13371333
custom_args=tool_kwargs,
13381334
function_calls=static_approved,
13391335
tools=tools,
13401336
middleware_pipeline=middleware_pipeline,
13411337
config=config,
13421338
)
1343-
approved_function_results = list(results)
13441339
except Exception as e:
13451340
logger.exception("Failed to execute approved tool calls; injecting error results: %s", e)
1346-
approved_function_results = []
1341+
approved_function_result_groups = []
13471342

1348-
# Build results for executed (static) approved responses (used for TOOL_CALL_RESULT event emission)
1343+
# Normalize one group per static approval and collect only terminal results for TOOL_CALL_RESULT events.
13491344
# Deferred provider-injected approvals are left in messages for ToolApprovalMiddleware to process.
1345+
replacement_groups: list[list[Content]] = []
13501346
approved_results: list[Content] = []
13511347
for idx, approval in enumerate(static_approved):
1352-
if (
1353-
idx < len(approved_function_results)
1354-
and getattr(approved_function_results[idx], "type", None) == "function_result"
1355-
):
1356-
approved_results.append(approved_function_results[idx])
1357-
continue
1358-
# Get call_id from function_call if present, otherwise use approval.id
1359-
func_call = approval.function_call
1360-
call_id = (func_call.call_id if func_call else None) or approval.id or ""
1361-
approved_results.append(
1362-
Content.from_function_result(call_id=call_id, result="Error: Tool call invocation failed.")
1363-
)
1364-
1365-
_replace_approval_contents_with_results(messages, fcc_todo, [[result] for result in approved_results])
1348+
result_group = approved_function_result_groups[idx] if idx < len(approved_function_result_groups) else []
1349+
if not result_group:
1350+
func_call = approval.function_call
1351+
call_id = (func_call.call_id if func_call else None) or approval.id or ""
1352+
result_group = [Content.from_function_result(call_id=call_id, result="Error: Tool call invocation failed.")]
1353+
replacement_groups.append(result_group)
1354+
approved_results.extend(content for content in result_group if content.type == "function_result")
1355+
1356+
_replace_approval_contents_with_results(messages, fcc_todo, replacement_groups)
13661357

13671358
# Post-process: Convert user messages with function_result content to proper tool messages.
13681359
# After _replace_approval_contents_with_results, approved tool calls have their results

0 commit comments

Comments
 (0)