Skip to content

openai-completions: chat-completions refusal field dropped, assistant turn resolves to empty content #102321

Description

@yetval

Summary

When an OpenAI chat-completions response carries a safety or structured-output refusal, the assistant turn resolves to empty content and the user sees nothing. OpenAI returns refusals in a top-level refusal field with content: null and finish_reason: "stop", but the chat-completions streaming assembler never reads refusal, so the refusal text is silently dropped.

Environment

  • Commit: aad9974 (origin/main at time of filing)
  • Surface: OpenAI-compatible chat-completions transport (openai-completions API), both streaming delta and aggregated message forms

Steps to reproduce

  1. Use a provider on the openai-completions API.
  2. Send a prompt the model refuses. OpenAI returns a chunk like {"choices":[{"delta":{"role":"assistant","refusal":"I can't help with that."},"finish_reason":"stop"}]} (or the aggregated {"message":{"role":"assistant","content":null,"refusal":"..."}}).
  3. Observe the resolved assistant turn.

Expected

The refusal is surfaced to the user. The same file already does this on the Responses path: response.refusal.delta is routed to the visible text block and output_item.done reads contentPart.refusal (src/agents/openai-transport-stream.ts lines 1635 and 1697-1700). The completions path should likewise surface the refusal rather than dropping it.

Actual

The assistant turn resolves to empty content (content: [], visible text ""). No text, no error, no diagnostic. The user sees a blank reply.

Root cause

src/agents/openai-transport-stream.ts, processOpenAICompletionsStream delta loop (around lines 3151-3183). The loop reads content, reasoning deltas, and tool_calls from choice.delta ?? choice.message, but never reads the top-level refusal field:

const choiceDelta =
  choice.delta ??
  (choice as unknown as { message?: ChatCompletionChunk["choices"][number]["delta"] }).message;
...
if (choiceDelta.content) {
  // routes visible text and thinking
}
// reasoningDeltas handled
// choiceDelta.tool_calls handled
// choiceDelta.refusal is never read

When content is null and only refusal is present, nothing is appended to output.content, so the turn is empty.

Sibling surfaces

  • Confirmed sibling: packages/ai/src/providers/openai-completions.ts streamOpenAICompletions (registered live via packages/ai/src/providers/register-builtins.ts) is a second completions streaming assembler that also has no refusal handling (no occurrence of refusal in the file). Its Responses counterpart is being taught refusal handling separately (see novelty note), but the completions path there drops it the same way.
  • Not affected: the Responses path in src/agents/openai-transport-stream.ts already routes refusal (lines 1635, 1697-1700, 1878).

Novelty

Distinct from the existing refusal reports, all of which concern the Responses path and/or mapping refusal to an error or fallback, not silent content-drop on chat-completions:

Real behavior proof

Behavior addressed: OpenAI chat-completions refusal-only response resolves to empty assistant content instead of surfacing the refusal.
Real environment tested: drove the real processOpenAICompletionsStream assembler (src/agents/openai-transport-stream.ts, exported via testing) end to end, stubbing only the transport input as an async iterable of chunks and keeping the assembly logic real, at commit aad9974.
Exact steps or command run after this patch: fed the assembler a streaming refusal chunk {choices:[{delta:{role:"assistant",refusal:"I can't help with that."},finish_reason:"stop"}]} and, separately, the aggregated {choices:[{message:{role:"assistant",content:null,refusal:"I can't help with that."},finish_reason:"stop"}]}, then read the resolved output.content.
Evidence after fix:

# OBSERVED (buggy, origin/main aad99747) - streaming delta form
content blocks: []
visible text: ""
# OBSERVED (buggy, origin/main aad99747) - aggregated message form
content blocks: []
visible text: ""

# EXPECTED (after reading refusal like the Responses path in the same file, identical inputs) - streaming delta form
content blocks: [{"type":"text","text":"I can't help with that."}]
visible text: "I can't help with that."
# EXPECTED (identical inputs) - aggregated message form
content blocks: [{"type":"text","text":"I can't help with that."}]
visible text: "I can't help with that."

Observed result after fix: on current main the refusal is dropped and the turn is empty; reading refusal (as the Responses path already does) surfaces the refusal string as visible assistant content on identical inputs.
What was not tested: not exercised against a live OpenAI endpoint or full CLI turn (the provider HTTP boundary was stubbed at the chunk iterable seam); did not evaluate whether the preferred surfacing is visible text versus a provider-refusal error/diagnostic; the packages/ai streamOpenAICompletions sibling was verified by source inspection, not driven in this repro.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.no-staleExclude from stale automation

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions