Python: [Feature]: Bound an agent run by duration (and by spend), not only by iteration and call count #7587

Description

Problem

FunctionInvocationConfiguration has exactly seven fields — enabled, max_iterations, max_function_calls, max_consecutive_errors_per_request, terminate_on_unknown_calls, additional_tools, include_detailed_errors. None of them expresses time, and none expresses cost.

The docstring nominates one of them as the cost control: "max_function_calls: Limits the total number of individual function invocations ... This is the primary knob for controlling cost and preventing runaway tool usage." A call count is not a cost. Two function calls can differ by three orders of magnitude in what they add to the conversation — one returns a boolean, the next returns a large file listing that is then re-sent on every subsequent iteration. So the knob documented as the cost control does not measure cost, and a run can sit comfortably inside both counters while spending without bound.

Concretely, from a deployed multi-agent application: a read-only reviewer agent ran 508 function calls against a limit of 1000, inside a roundtrip budget of 200, while consuming roughly 6.5M input tokens over ~100 minutes in a single run. Both counters were satisfied for the whole window. The bound that mattered — "this has been going for over an hour, wrap up with what you have" — cannot be stated in this configuration at all.

There is no caller-side answer either, for a run that is not awaited inline:

• Agent.run takes no timeout parameter (messages, stream, session, middleware, tools, options, compaction_strategy, tokenizer, function_invocation_kwargs, client_kwargs).
• Wrapping the call in asyncio.wait_for cancels the coroutine mid-tool-call. The partial work is lost and the model never gets to produce a final response — materially worse than what the existing budgets do, which is to disable the tools and ask the model to answer (_disable_tools_at_function_call_limit, _tools.py:2935 and :2982). That is a graceful landing that keeps everything the run has produced so far.

The asymmetry is the whole request: the library already knows how to stop a runaway gracefully, and that mechanism simply cannot be triggered by a clock.

Requested

1. max_duration_seconds in FunctionInvocationConfiguration, checked at the top of the tool loop (_tools.py:2938) beside the existing iteration bound, and on expiry taking the same path as the call limit: tools disabled, model asked for a final response, and a log line in the shape of the existing "Maximum iterations reached (%d). Requesting final response without tools." (:2990). Best-effort in exactly the sense already documented for max_function_calls — evaluated between batches, never interrupting one in flight.
2. A cumulative input-token bound per request, if the axis is worth exposing directly. Roundtrips and call counts are both proxies for spend, and both proxies fail in the same direction: a long-context run is expensive per unit of every counter the library currently offers.
3. A machine-readable signal that a bound stopped the run, whichever bounds exist. Today a caller cannot distinguish "the model finished" from "the model was stopped and asked to summarize", and the difference matters to anything that decides whether to continue. We hit the consequence directly: a model whose tools went quiet at a budget wall, with no notice explaining it, confabulated a cause — it reported having lost write access to its workspace, and the operator-visible symptom was a task that produced no files for a reason that never happened. We now inject an explicit notice at that boundary for exactly this reason; a framework-level flag would make it unnecessary.

Evidence it matters

• The 100-minute, 6.5M-token reviewer run above: inside every configured budget for its whole duration, and stopped by nothing except eventually finishing. The supervising agent that was polling it spent a further ~4M input tokens narrating its progress across ~30 polls.
• The confabulated-cause failure in request 3 is not hypothetical: it is why our application carries a budget-exhaustion notice at all.

Workaround (application-side, today)

maf_governor.RunDeadlineMiddleware: a FunctionMiddleware that records a start time per run and, once the deadline has passed, answers every subsequent tool call with a short notice instead of executing it, so the model runs out of moves and produces its final response. It composes with the graceful-landing behavior above rather than fighting it.

What it cannot do, and why the loop-level version is strictly better: a middleware only runs when a tool is called. A model that spends ten minutes inside a single reasoning-heavy roundtrip, or that stops calling tools and simply keeps generating, is not observed by this shim until its next tool call — which may never come. A check at the top of the tool loop has no such blind spot, and it is the only place a wall-clock bound can be enforced honestly.

Code Sample

Language/SDK

Python