Skip to content

[Bug]: Non-streaming Responses get_response treats response.failed/incomplete as a successful empty turn #4515

Closed

Description

@weike-zhang

Expected Behavior

When the Responses API reports a terminal failed or incomplete response (e.g. status="incomplete" with incomplete_details.reason="max_output_tokens"), the model layer should not treat it as a successful empty turn. Every other path already rejects these terminal states as ModelBehaviorError:

  • stream_response() (HTTP streaming) — raises response_terminal_failure_error(...) for response.failed / response.incomplete events
  • Websocket get_response() — raises response_terminal_failure_error(...) for the same terminal payloads
  • Streamed runner finalization — converted in run_loop.py

This family was fixed in #3106 (closed 2026-05-04) — but the HTTP non-streaming get_response() path was never covered.

Current Behavior

OpenAIResponsesModel.get_response() (HTTP, non-streaming) returns ModelResponse(output=[]) when client.responses.create() returns a Response with status="incomplete" or status="failed". AnyLLMModel._get_response_via_responses() has the same gap.

A run that hits max_output_tokens before emitting any visible token therefore concludes as an indistinguishable empty turn instead of surfacing the truncation — the same failure mode as #4510 (which was fixed for the chat-completions path), and structured-output runs can fall into the same empty-output retry loop.

Steps to Reproduce

import asyncio
from typing import Any, cast
from openai.types.responses import Response
from openai.types.responses.response import IncompleteDetails
from agents import ModelSettings
from agents.models.openai_responses import OpenAIResponsesModel
from agents.tracing.model_tracing import ModelTracing

class Responses:
    async def create(self, **kwargs: Any) -> Response:
        return Response(
            id="resp-incomplete",
            created_at=0, model="fake", object="response", output=[],
            tool_choice="none", tools=[], top_p=None, parallel_tool_calls=False,
            usage=None,
            status="incomplete",
            incomplete_details=IncompleteDetails(reason="max_output_tokens"),
        )

class Client:
    responses = Responses()

async def main() -> None:
    model = OpenAIResponsesModel(model="gpt-4", openai_client=cast(Any, Client()))
    result = await model.get_response(
        system_instructions=None, input="hi", model_settings=ModelSettings(),
        tools=[], output_schema=None, handoffs=[], tracing=ModelTracing.DISABLED,
    )
    print("output =", result.output)   # [] -- treated as a successful empty turn

asyncio.run(main())

Output:

output = []

No error is raised. (Control: the same mocked response on status="completed" also returns []; the streaming and websocket paths raise ModelBehaviorError for failed/incomplete.)

Root Cause

  • src/agents/models/openai_responses.py → _fetch_response() (non-stream branch) does not check response.status.
  • src/agents/extensions/models/any_llm_model.py → _get_response_via_responses() does not check response.status.

The terminal-state handling introduced for #3106 only covered the event-driven paths (streaming + websocket). The HTTP non-streaming path returns the raw Response object, whose terminal status is silently ignored.

Suggested Fix

Raise the same ModelBehaviorError used elsewhere when the non-streaming response is terminal-failed:

if getattr(response, "status", None) in {"failed", "incomplete"}:
    raise response_terminal_failure_error(f"response.{response.status}", response)

in both _fetch_response (non-stream) and _get_response_via_responses. This makes non-streaming behavior consistent with the streaming and websocket paths.

Impact

Non-streaming runs (the default for Runner.run) that hit max_output_tokens with zero emitted tokens surface as an empty turn with no explanation, and structured-output runs can loop on empty output (see #4510 for the same loop on the chat-completions path). The fix is a two-line consistency change.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions