-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
openai-completions: chat-completions refusal field dropped, assistant turn resolves to empty content #102321
Copy link
Copy link
Closed
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.ClawSweeper marked this issue as an existing queue_fix_pr work candidate.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.Auth, provider routing, model choice, or SecretRef resolution may break.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.Channel message delivery can be lost, duplicated, or misrouted.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.no-staleExclude from stale automationExclude from stale automation
Description
Metadata
Metadata
Assignees
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.ClawSweeper marked this issue as an existing queue_fix_pr work candidate.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.Auth, provider routing, model choice, or SecretRef resolution may break.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.Channel message delivery can be lost, duplicated, or misrouted.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.no-staleExclude from stale automationExclude from stale automation
Type
Fields
Priority
None yet
Summary
When an OpenAI chat-completions response carries a safety or structured-output refusal, the assistant turn resolves to empty content and the user sees nothing. OpenAI returns refusals in a top-level
refusalfield withcontent: nullandfinish_reason: "stop", but the chat-completions streaming assembler never readsrefusal, so the refusal text is silently dropped.Environment
openai-completionsAPI), both streaming delta and aggregated message formsSteps to reproduce
openai-completionsAPI.{"choices":[{"delta":{"role":"assistant","refusal":"I can't help with that."},"finish_reason":"stop"}]}(or the aggregated{"message":{"role":"assistant","content":null,"refusal":"..."}}).Expected
The refusal is surfaced to the user. The same file already does this on the Responses path:
response.refusal.deltais routed to the visible text block andoutput_item.donereadscontentPart.refusal(src/agents/openai-transport-stream.ts lines 1635 and 1697-1700). The completions path should likewise surface the refusal rather than dropping it.Actual
The assistant turn resolves to empty content (
content: [], visible text""). No text, no error, no diagnostic. The user sees a blank reply.Root cause
src/agents/openai-transport-stream.ts,
processOpenAICompletionsStreamdelta loop (around lines 3151-3183). The loop readscontent, reasoning deltas, andtool_callsfromchoice.delta ?? choice.message, but never reads the top-levelrefusalfield:When
contentis null and onlyrefusalis present, nothing is appended tooutput.content, so the turn is empty.Sibling surfaces
streamOpenAICompletions(registered live via packages/ai/src/providers/register-builtins.ts) is a second completions streaming assembler that also has norefusalhandling (no occurrence ofrefusalin the file). Its Responses counterpart is being taught refusal handling separately (see novelty note), but the completions path there drops it the same way.refusal(lines 1635, 1697-1700, 1878).Novelty
Distinct from the existing refusal reports, all of which concern the Responses path and/or mapping refusal to an error or fallback, not silent content-drop on chat-completions:
stopReason: "error"; does not touch the completions assembler.This report is the chat-completions content-drop-to-empty case, which none of the above cover.
Real behavior proof
Behavior addressed: OpenAI chat-completions refusal-only response resolves to empty assistant content instead of surfacing the refusal.
Real environment tested: drove the real
processOpenAICompletionsStreamassembler (src/agents/openai-transport-stream.ts, exported viatesting) end to end, stubbing only the transport input as an async iterable of chunks and keeping the assembly logic real, at commit aad9974.Exact steps or command run after this patch: fed the assembler a streaming refusal chunk
{choices:[{delta:{role:"assistant",refusal:"I can't help with that."},finish_reason:"stop"}]}and, separately, the aggregated{choices:[{message:{role:"assistant",content:null,refusal:"I can't help with that."},finish_reason:"stop"}]}, then read the resolvedoutput.content.Evidence after fix:
Observed result after fix: on current main the refusal is dropped and the turn is empty; reading
refusal(as the Responses path already does) surfaces the refusal string as visible assistant content on identical inputs.What was not tested: not exercised against a live OpenAI endpoint or full CLI turn (the provider HTTP boundary was stubbed at the chunk iterable seam); did not evaluate whether the preferred surfacing is visible text versus a provider-refusal error/diagnostic; the packages/ai
streamOpenAICompletionssibling was verified by source inspection, not driven in this repro.