Skip to content

[Bug] output_model pops the pending tool-call message on paused (HITL) or stop_after_tool_call runs, corrupting the persisted transcript and continue payload #11014

Open

Description

@BlueX888

Description

get_messages_for_output_model() (libs/agno/agno/agent/_messages.py:2513) pops the last message of the run transcript unconditionally before the output model is invoked. The pop is meant to drop the main model's draft answer so the output model can restructure it, but when an agent has both output_model and a tool that pauses the run (requires_confirmation=True, requires user input, or external execution) — or a tool that sets stop_after_tool_call — the transcript's last message at that point is instead the assistant message carrying the pending tool_calls (pause path) or the tool result (stop path). Output-model generation runs before the paused-check at every call site (_run.py:582 → paused-check at 599, and the stream/async/continue equivalents), so popping it:

  • persists the paused run with no record of the pending tool call, and its content becomes an answer fabricated by the output model before the tool ever ran;
  • on continue_run, the executed tool's result is appended to that same transcript, so the payload sent to the main model contains an orphaned tool message with no preceding assistant tool_calls — OpenAI/Anthropic/Gemini reject such payloads with a 400;
  • with stop_after_tool_call, the pop removes the tool result instead: the persisted transcript keeps 0 tool results and the output model is called with an orphaned assistant tool_call message as its last message.

Steps to Reproduce

Self-contained script (stub Model subclass — no API keys, no network), run against an editable install of agno at commit 9e1f77f:

from typing import Any, AsyncIterator, Iterator, List

from agno.agent.agent import Agent
from agno.db.in_memory import InMemoryDb
from agno.models.base import Model
from agno.models.message import MessageMetrics
from agno.models.response import ModelResponse
from agno.run.agent import RunOutput, ToolExecution
from agno.run.requirement import RunRequirement
from agno.tools.function import Function


class StubModel(Model):
    """main: 1st invoke -> get_weather tool call, later invokes -> text. output: always text."""

    def __init__(self, name: str, text: str = "", tool_call: bool = False):
        super().__init__(id=name, name=name, provider="test")
        self.text, self.tool_call, self.calls = text, tool_call, 0
        self.received: List[List[Any]] = []

    # --- boilerplate so Model works without a real provider ---
    def get_instructions_for_model(self, *a, **k): return None
    def get_system_message_for_model(self, *a, **k): return None
    async def aget_instructions_for_model(self, *a, **k): return None
    async def aget_system_message_for_model(self, *a, **k): return None
    def parse_args(self, *a, **k): return {}
    def _parse_provider_response(self, response: Any, **kwargs) -> ModelResponse:
        return ModelResponse(content="x", role="assistant")
    def _parse_provider_response_delta(self, response: Any) -> ModelResponse:
        return ModelResponse(content="x", role="assistant")

    def invoke(self, *args, messages=None, **kwargs) -> ModelResponse:
        self.received.append(list(messages or []))
        self.calls += 1
        if self.tool_call and self.calls == 1:
            return ModelResponse(
                content="", role="assistant",
                tool_calls=[{"id": "call_1", "type": "function",
                             "function": {"name": "get_weather", "arguments": '{"city": "Paris"}'}}],
                response_usage=MessageMetrics(),
            )
        return ModelResponse(content=self.text, role="assistant", response_usage=MessageMetrics())

    async def ainvoke(self, *args, **kwargs) -> ModelResponse:
        return self.invoke(*args, **kwargs)

    def invoke_stream(self, *args, **kwargs) -> Iterator[ModelResponse]:
        yield self.invoke(*args, **kwargs)

    async def ainvoke_stream(self, *args, **kwargs) -> AsyncIterator[ModelResponse]:
        yield await self.ainvoke(*args, **kwargs)


def get_weather(city: str) -> str:
    return f"The weather in {city} is sunny."


weather = Function.from_callable(get_weather)
weather.requires_confirmation = True  # HITL: run pauses and waits for confirmation

main_model = StubModel("main", tool_call=True)
agent = Agent(
    model=main_model,
    output_model=StubModel("output", text="FINAL ANSWER (tool never ran!)"),
    tools=[weather],
    db=InMemoryDb(),
)

resp: RunOutput = agent.run(input="weather in Paris?")
print("run 1 status:", resp.status, "| content:", repr(resp.content))
for m in resp.messages:
    print(f"  persisted: role={m.role!r} tool_calls={'yes' if m.tool_calls else 'no'} content={str(m.content)[:45]!r}")

confirmed = ToolExecution(
    tool_call_id=resp.tools[0].tool_call_id, tool_name="get_weather",
    tool_args={"city": "Paris"}, requires_confirmation=True, confirmed=True,
)
agent.continue_run(run_id=resp.run_id, requirements=[RunRequirement(tool_execution=confirmed)])

print("payload sent to the main model on the post-continue call:")
for m in main_model.received[-1]:
    print(f"  payload: role={m.role!r} tool_calls={'yes' if m.tool_calls else 'no'} "
          f"tool_call_id={getattr(m, 'tool_call_id', None)!r} content={str(m.content)[:45]!r}")

Run: python repro.py

Agent Configuration (if applicable)

Embedded in the script above: Agent(model=..., output_model=..., tools=[Function with requires_confirmation=True], db=InMemoryDb()), run via agent.run() then agent.continue_run(...).

Expected Behavior

  • The pop is meant to remove the main model's draft answer — the function's own comment says "Remove the last assistant message from the messages list" (_messages.py:2512), and output_model exists to "structure the response from the main model" (agent.py:323). On the pause path the last message is not a draft answer: it is the assistant message bearing the pending tool_calls, and it must survive so the paused run can be resumed.
  • The control run without output_model shows the intended shape: the paused run is persisted with the assistant tool_calls message, and after confirmation the main model receives a valid [user, assistant(tool_calls), tool] payload (output below).
  • handle_agent_run_paused (_run.py:236-257) persists the paused run precisely so it can be resumed — "We return and await confirmation/completion for the tools that require it" (_run.py:256) — and on continue_run the confirmed tool's result is appended to that same transcript via run_tool (_tools.py:1088). That only produces a valid provider payload if the pending assistant tool_call message is still present.
  • The team-level counterpart of this exact function is already hardened against both problems: _get_messages_for_output_model (libs/agno/agno/team/_messages.py:1585) deepcopies the messages "to avoid mutating the originals" and only pops when messages[-1].role == "assistant" (team/_messages.py:1607-1608).

Actual Behavior

With output_model set and a requires_confirmation=True tool:

  1. run 1 returns RunStatus.paused but with content='FINAL ANSWER (tool never ran!)' — produced by the output model before the tool was confirmed or executed.
  2. The persisted paused-run transcript is [user, assistant] where the assistant message has no tool_calls — the pending tool call is erased from storage.
  3. After continue_run confirms the tool, the payload sent to the main model is [user, assistant(no tool_calls), tool(call_1)] — an orphaned tool message that providers reject with a 400 (No tool call found for tool call id ...).

With stop_after_tool_call (same root cause): the pop removes the tool result instead — the persisted transcript keeps 0 tool result messages, and the output model is called with an orphaned assistant tool_call message as its last message.

Screenshots or Logs (if applicable)

Verbatim output of the script above:

run 1 status: RunStatus.paused | content: 'FINAL ANSWER (tool never ran!)'
  persisted: role='user' tool_calls=no content='weather in Paris?'
  persisted: role='assistant' tool_calls=no content='FINAL ANSWER (tool never ran!)'
payload sent to the main model on the post-continue call:
  payload: role='user' tool_calls=no tool_call_id=None content='weather in Paris?'
  payload: role='assistant' tool_calls=no tool_call_id=None content='FINAL ANSWER (tool never ran!)'
  payload: role='tool' tool_calls=no tool_call_id='call_1' content='The weather in Paris is sunny.'

Control run (identical flow, output_model=None) keeps the tool-call message and produces a valid post-continue payload:

run 1 status: RunStatus.paused | content: 'I have tools to execute, but I need confirmation.'
run 1 persisted transcript:
  [0] role='user' tool_calls=no content='weather in Paris?'
  [1] role='assistant' tool_calls=yes content=''
run 2 status: RunStatus.completed
provider payload for the post-continue main-model call:
  [0] role='user' tool_calls=no tool_call_id=None content='weather in Paris?'
  [1] role='assistant' tool_calls=yes tool_call_id=None content=''
  [2] role='tool' tool_calls=no tool_call_id='call_1' content='The weather in Paris is sunny.'
orphaned tool messages in payload: []

while the output_model run reports orphaned tool messages in payload: ['call_1'] (same script with an explicit orphan check). An independent verification run confirmed both: output_model=True: run 1 status: RunStatus.paused | content: 'FINAL ANSWER (tool never ran!)'; transcript [1] role='assistant' tool_calls=no; post-continue payload: [user, assistant, tool] -> orphaned tool messages in payload: ['call_1'] (control, output_model=False: transcript [1] role='assistant' tool_calls=yes; orphaned: []). For the stop_after_tool_call variant the measured result is tool result messages kept in transcript: 0.

Environment

  • OS: macOS 26.6 (arm64)
  • Agno Version: main @ 9e1f77f52a9bb16ee5dde3ccff9fa2491e86db5f (pyproject 3.1.2), editable install
  • External Dependency Versions: none needed for the repro (stub Model subclass, no API keys)
  • Additional Environment Details: Python 3.12.14

Possible Solutions (optional)

Two candidate directions (either would fix the reported paths):

  1. In get_messages_for_output_model (libs/agno/agno/agent/_messages.py:2513), only pop when the last message is an assistant message carrying no tool_calls, and avoid mutating the caller's list in place — the team-level counterpart (team/_messages.py:1585) already does both.
  2. Skip output-model generation when the run has paused tools or stop_after_tool_call fired — i.e. move the generate_response_with_output_model call after the is_paused check at the call sites (_run.py:582 vs 599, and the stream/async/continue equivalents at 1029/1066, 1733/1756, 2381/2426, 3733/3750, 4974/4997, 5524/5570).

I'm happy to open a PR with a fix and a regression test once the issue is triaged.

Additional Context

  • Related: OpenAI Responses: orphaned function_call_output (result without its call) still poisons sessions #9372 reports the same symptom class (an orphaned tool-result message in a provider payload → 400), but from a different layer and trigger — the OpenAI Responses adapter's _format_messages under history truncation. This issue is the agent transcript itself losing the pending tool-call message, which affects all providers.
  • Reachability: any Agent(output_model=...) with a tool that pauses (requires_confirmation=True, requires user input, external execution) or sets stop_after_tool_call, via run()/arun()/streaming and the matching continue paths.
  • Disclosure: this report was prepared with the help of an AI coding agent (repro script and root-cause analysis); the behavior was verified by running the script above against the commit listed in Environment and reviewed before filing.

Activity

  1. BlueX888 commented on Oct 11, 2026

    @BlueX888
    ContributorAuthor

    Reproduced this. Root cause: libs/agno/agno/agent/_messages.py:2513 — get_messages_for_output_model() pops the transcript's last message unconditionally, so on a paused run it erases the assistant message carrying the pending tool_calls (and on stop_after_tool_call it erases the tool result instead).

    Minimal reproduction:

    python repro.py
    

    Observed output:

    run 1 status: RunStatus.paused | content: 'FINAL ANSWER (tool never ran!)'
      persisted: role='user' tool_calls=no content='weather in Paris?'
      persisted: role='assistant' tool_calls=no content='FINAL ANSWER (tool never ran!)'
    payload sent to the main model on the post-continue call:
      payload: role='assistant' tool_calls=no tool_call_id=None content='FINAL ANSWER (tool never ran!)'
      payload: role='tool' tool_calls=no tool_call_id='call_1' content='The weather in Paris is sunny.'
    

    Happy to open a PR with this fix, or to be assigned if you prefer to take it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions