Skip to content

Streamed tool-call turns from Chat Completions providers poison the session: replay produces an invalid message sequence and every later turn 400s #4727

Closed

Description

@mmjerge

Please read this first

Describe the bug

A Chat Completions provider that streams tool-call deltas before the same turn's text produces a turn the SDK cannot replay. The first turn succeeds. Every turn after it fails with 400. The session is unrecoverable.

The stream handler preserves stream order in the response outputs: [function_call, message]. On the next turn Converter.items_to_messages replays that history and flushes the pending tool_calls message when it meets the message item, emitting two adjacent assistant messages:

[0] assistant {tool_calls: [call_abc]}
[1] assistant "Let me check the weather."
[2] tool call_abc "72F"

OpenAI rejects this: "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'." Anthropic-style endpoints enforce the same adjacency.

The identical turn converts correctly when not streamed, because the non-streaming path orders outputs [message, function_call], which folds into a single assistant message. Same model, same turn: non-streamed works, streamed poisons the session — including histories already persisted to Sessions, which fail on every replay.

Debug information

  • Agents SDK version: main at e773b15; also v0.22.0
  • Python version: 3.10.16 (repo default via uv)
  • Operating system: macOS (darwin, arm64)
  • Model and model provider: any Chat Completions provider that streams tool-call deltas before same-turn text (observed via LiteLLM-routed providers); reproduces without network using the stream handler directly
  • Does the issue reproduce with the latest Agents SDK release? Yes, and on main.
  • Does the issue occur consistently or intermittently? Deterministic given that stream ordering.

Repro steps

Self-contained, no API key needed — simulates the provider stream, then replays the resulting history:

import asyncio
from openai.types.chat import ChatCompletionChunk
from openai.types.chat.chat_completion_chunk import (
    Choice, ChoiceDelta, ChoiceDeltaToolCall, ChoiceDeltaToolCallFunction,
)
from openai.types.responses import Response
from agents.models.chatcmpl_stream_handler import ChatCmplStreamHandler
from agents.models.chatcmpl_converter import Converter

def chunk(delta, finish=None):
    return ChatCompletionChunk(id="c", created=0, model="m",
        object="chat.completion.chunk",
        choices=[Choice(index=0, delta=delta, finish_reason=finish)])

async def fake_stream():
    yield chunk(ChoiceDelta(tool_calls=[ChoiceDeltaToolCall(index=0, id="call_abc",
        type="function", function=ChoiceDeltaToolCallFunction(name="get_weather", arguments=""))]))
    yield chunk(ChoiceDelta(tool_calls=[ChoiceDeltaToolCall(index=0,
        function=ChoiceDeltaToolCallFunction(arguments='{"city":"SF"}'))]))
    yield chunk(ChoiceDelta(content="Let me check the weather."))
    yield chunk(ChoiceDelta(), finish="tool_calls")

async def main():
    base = Response(id="r", created_at=0, model="m", object="response", output=[],
                    tool_choice="auto", tools=[], parallel_tool_calls=False)
    events = [e async for e in ChatCmplStreamHandler.handle_stream(base, fake_stream())]
    final = [e for e in events if e.type == "response.completed"][0].response
    items = [o.model_dump(exclude_unset=True) for o in final.output]
    items.append({"type": "function_call_output", "call_id": "call_abc", "output": "72F"})
    msgs = Converter.items_to_messages(items)
    for i, m in enumerate(msgs):
        print(i, m["role"], "tool_calls" if m.get("tool_calls") else "", m.get("content"))

asyncio.run(main())

Output on main: two adjacent assistant messages, the first with tool_calls. Sent upstream, that is a 400.

Expected behavior

The streamed turn and the non-streamed turn describe the same thing and must convert to the same messages: one assistant message carrying both the text and the tool_calls, followed by the tool message.

A converter-side fix also heals histories already persisted in sessions, which a stream-handler-side fix would not.

PR with a fix and regression tests is up alongside this issue.

Activity

  1. mmjerge commented on Aug 28, 2026

    @mmjerge
    ContributorAuthor

    Fix with regression tests: #4728

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions