You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 279615e
Browse filesBrowse the repository at this point in the historyBrowse files
- Remove agent.as_tool usage rollup from budget accounting (as_tool API deprecated, microsoft#8950)
- Keep BackgroundAgentsProvider rollup: max_tokens is a thinking budget — a uniform
bound on total inference work — not a cost proxy; cross-model rollup is intentional
- Fix max_duration: now interrupts in-flight model/tool calls via asyncio.wait_for,
not only between iterations
- Clarify AgentBudget.max_tokens docstring: thinking budget vs cost estimate
- Partially addresses eavanvalkenburg feedback on microsoft#8862; see issue for our rationale
on keeping sub-agent token accounting
Generated with Amplifier
Co-Authored-By: Amplifier <[email protected]>
Copy file name to clipboardExpand all lines: python/packages/core/AGENTS.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -309,7 +309,7 @@ The vector store API is experimental under the shared `VECTOR_STORES` feature ID
309
309
- **`todos_remaining(*, looping_modes=None)`** / **`todos_remaining_message`** - Helper factories for todo-driven loops (the Python counterpart of .NET's `TodoCompletionLoopEvaluator`), designed for `create_harness_agent` but usable with any agent that registers a `TodoProvider` via `context_providers`. They resolve the `TodoProvider`/`AgentModeProvider` from the *running agent* (`agent.context_providers`, via `_resolve_context_provider`) rather than taking the provider as an argument, so they can be wired directly into `loop_should_continue`/`loop_next_message`. `todos_remaining` returns a `should_continue` predicate that loops while any todo is open; pass `looping_modes=[...]` to gate looping to specific operating modes (case-insensitive; honors the `AgentModeProvider`'s `source_id`/`available_modes`), `looping_modes=None` (default) applies in every mode, and an empty sequence raises `ValueError`. `todos_remaining_message` is a `next_message` callable that lists the still-open todo titles and tells the agent to finish them, returning `None` when the session/agent/provider is unavailable or nothing is open (in which case the middleware's default `None` handling applies: reuse the previous iteration's messages verbatim under the default `fresh_context=False`, or `DEFAULT_NEXT_MESSAGE` only when `fresh_context=True`).
310
310
-**`background_tasks_running()`** / **`background_tasks_running_message`** - Helper factories for background-agent-driven loops, mirroring the `todos_remaining` pair. They resolve the `BackgroundAgentsProvider` from the *running agent* (`agent.context_providers`, via `_resolve_context_provider`) rather than taking the provider as an argument, so they can be wired directly into `create_harness_agent`'s `loop_should_continue`/`loop_next_message`. `background_tasks_running` returns a `should_continue` predicate that loops while the provider's persisted state shows any task with `status == RUNNING` (pair it with `max_iterations` so the loop is bounded even if a task's persisted status is never refreshed). `background_tasks_running_message` is a `next_message` callable that lists the still-running tasks (`#<id> (<agent_name>): <description>`) and tells the agent to wait for them to finish and retrieve their results, returning `None` when the session/agent/provider is unavailable or no task is running.
311
311
-**Approval escape hatch** - `_has_pending_approval_request(result)` checks whether an iteration's response carries a pending tool-approval request (any content with `type == "function_approval_request"`). Both the streaming and non-streaming loops stop and return that response to the caller *before* evaluating `should_continue`/`max_iterations` or injecting `next_message`, so the loop is HITL-safe even when wrapped outermost around a `ToolApprovalMiddleware` (mirrors the C# `LoopAgent`'s `HasPendingApprovalRequests`).
312
-
- **Budget** - `budget=AgentBudget` (a `total=False` `TypedDict`; plain dicts work) adds `max_tokens` / `max_duration` limits, validated in `AgentLoopMiddleware.__init__` and checked after each iteration before `should_continue`. `max_tokens` charges each iteration the wrapped agent response's `total_token_count`, falling back to `input_token_count + output_token_count`; the `with_judge` judge call is never charged. Sub-agent usage counts: `as_tool` reports the sub-agent's `usage_details` through a private `FunctionInvocationContext.metadata` key (`_TOOL_USAGE_CONTEXT_KEY`), which the function invocation loop moves onto the function_result and folds into the parent response exactly once (non-streaming `aggregated_usage`; streaming via a `usage` content on the tool update), so nested A->B->C counts each layer once. `BackgroundAgentsProvider` stores each finished run's usage on `BackgroundTaskInfo` (`usage_details`/`usage_reported`) and charges it once, to the parent run that first observes the completion (via the observing tool's context, or a run-scoped chat middleware when `before_run` observes it). `invoke_agent` telemetry spans still report each agent's own usage. Gaps: failed/cancelled sub-agent runs produce no response (tokens lost), background tasks never observed as finished or abandoned at `release_session` are not counted, and custom tools that call `agent.run` directly are not counted. The accounting is not configurable.
312
+
- **Budget** - `budget=AgentBudget` (a `total=False` `TypedDict`; plain dicts work) adds `max_tokens` / `max_duration` limits, validated in `AgentLoopMiddleware.__init__` and checked after each iteration before `should_continue`. `max_tokens` is a *thinking budget* (a uniform bound on total inference work, not a cost estimate): each iteration is charged the wrapped agent response's `total_token_count`, falling back to `input_token_count + output_token_count`; the `with_judge` judge call is never charged. `agent.as_tool` usage is NOT rolled up (API deprecated, #8950). `BackgroundAgentsProvider` rollup stays and is intentionally cross-model: it stores each finished run's usage on `BackgroundTaskInfo` (`usage_details`/`usage_reported`) and charges it once, to the parent run that first observes the completion (via the observing tool's context, or a run-scoped chat middleware when `before_run` observes it), using the private `_record_tool_usage` / `_TOOL_USAGE_CONTEXT_KEY` plumbing in `_tools.py`. `invoke_agent` telemetry spans still report each agent's own usage. Gaps: failed/cancelled sub-agent runs produce no response (tokens lost), background tasks never observed as finished or abandoned at `release_session` are not counted, and custom tools that call `agent.run` directly are not counted. The accounting is not configurable. `max_duration` is enforced between iterations and also interrupts the in-flight call via `asyncio.wait_for` (3.10-compatible); it re-raises `TimeoutError` if the first iteration times out with no prior result, otherwise stops cleanly with `loop_exit_reason="time_budget_exceeded"` (streaming: the timeout covers starting the stream, not consuming it).
313
313
-**Harness integration** - `create_harness_agent` enables the loop when a `loop_should_continue` callable is passed; it prepends `AgentLoopMiddleware(loop_should_continue, max_iterations=loop_max_iterations, next_message=loop_next_message)` ahead of `ToolApprovalMiddleware` so the loop is the outermost middleware (each iteration is a full agent run including tool approval, and the escape hatch hands pending approvals back to the caller). `loop_next_message` and `loop_max_iterations` only take effect together with `loop_should_continue` (with no `loop_should_continue` there is no loop, so they are ignored); `loop_max_iterations` defaults to the loop's default cap (`None` → unbounded).
0 commit comments