Repository navigation
fix(openai): keep Chat Completions reasoning on its assistant message - #1669
Conversation
Chat Completions providers that return reasoning on the assistant message are converted into a reasoning item followed by the assistant message item. Replaying that history emitted a separate assistant message with no content and no tool calls, and detached the reasoning from the message it described. Hand a reasoning-only pending assistant message to the assistant message that immediately follows it instead.
🦋 Changeset detectedLatest commit: 65b550b The changes in this PR will be included in the next version bump. This PR includes changesets to release 6 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The converter behavior is internally plausible, but we need evidence that this exact Chat Completions-compatible path occurs with a real provider before adding permanent replay behavior. Could you share the provider and model, the raw first-turn assistant message containing With a real-provider reproduction, we can reconsider this as a narrow round-trip fix. |
The reasoning-only assistant message was added once per reasoning turn, so a five turn conversation replayed eight assistant messages where four were enough. Pin the count and the reasoning placement across turns.
|
Fair ask. One correction first: I wrote that providers reject the payload. I had no basis for that and I withdraw it. No provider error is involved. Here is the live capture instead, and I have rewritten the description around it. Provider: OpenRouter, Raw first-turn assistant message, off the wire: {
"role": "assistant",
"content": "The capital of France is **Paris**.",
"refusal": null,
"reasoning": "Okay, the user asked a straightforward question: [truncated]",
"reasoning_details": [ { "type": "reasoning.text", "text": "[truncated]", "format": "unknown", "index": 0 } ]
}Second turn request from the same run: [
{ "role": "user", "content": "What is the capital of France?" },
{ "role": "assistant", "content": null, "reasoning": "[truncated]" },
{ "role": "assistant", "content": [ { "type": "text", "text": "The capital of France is **Paris**." } ] },
{ "role": "user", "content": "And Germany?" }
]The observable result. No error, OpenRouter answered normally both turns. What is wrong is that one provider message replays as two, the reasoning sits on the message with With the patch: 0, 1, 2, 3, 4 and never a contentless one. Same on the streaming path, which produces the same history items. So today a reasoning model on Chat Completions replays twice the assistant messages it needs, growing with conversation length, and the caller pays input tokens on the empty half. OpenRouter documents passing reasoning back on the assistant message, You are right that this is not an OpenAI field. The SDK already targets it deliberately on both sides though. Repro if you want to rerun it, needs an OpenRouter key and nothing else: import OpenAI from 'openai';
import { Agent, run, MemorySession, setTracingDisabled, OpenAIChatCompletionsModel } from '@openai/agents';
setTracingDisabled(true);
const client = new OpenAI({
apiKey: process.env.OPENROUTER_API_KEY,
baseURL: 'https://openrouter.ai/api/v1',
});
const requests = [];
const responses = [];
const create = client.chat.completions.create.bind(client.chat.completions);
client.chat.completions.create = async (body, options) => {
requests.push(structuredClone(body));
const response = await create(body, options);
responses.push(structuredClone(response));
return response;
};
const agent = new Agent({
name: 'Assistant',
instructions: 'Answer in one short sentence.',
model: new OpenAIChatCompletionsModel(client, 'deepseek/deepseek-r1'),
modelSettings: { providerData: { include_reasoning: true }, maxTokens: 1200 },
});
const session = new MemorySession();
await run(agent, 'What is the capital of France?', { session });
await run(agent, 'And Germany?', { session });
console.log(JSON.stringify(responses[0].choices[0].message, null, 2));
console.log(JSON.stringify(requests[1].messages, null, 2));I kept the change narrow on purpose since that is what you asked for. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 65b550bbf6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
An unknown item was pushed straight into the message list while an assistant message was still pending, so it landed out of history order and let a preceding reasoning item attach to an assistant message on the far side of it. Flush the pending assistant message first. Raised by Codex review on openai#1669.
seratch
left a comment
There was a problem hiding this comment.
Confirmed this works better with various model providers on openrouter
Summary
Chat Completions responses that carry reasoning do not round trip. Providers that return
reasoningon the assistant message get it recorded as a standalonereasoningitem placed before the assistantmessageitem, and on replay the converter turns that one provider message back into two assistant messages, with the reasoning on the one that has no content.Captured live against OpenRouter and
deepseek/deepseek-r1on@openai/[email protected]. Provider response,choices[0].message:{ "role": "assistant", "content": "The capital of France is **Paris**.", "refusal": null, "reasoning": "Okay, the user asked a straightforward question: [truncated]", "reasoning_details": [ { "type": "reasoning.text", "text": "[truncated]", "format": "unknown", "index": 0 } ] }Second turn request from the same run:
[ { "role": "user", "content": "What is the capital of France?" }, { "role": "assistant", "content": null, "reasoning": "[truncated]" }, { "role": "assistant", "content": [ { "type": "text", "text": "The capital of France is **Paris**." } ] }, { "role": "user", "content": "And Germany?" } ]It compounds, because a reasoning item is added per turn. Assistant messages in each outgoing request over five turns:
So a reasoning model on the Chat Completions path currently replays twice the assistant messages it needs, growing with conversation length, and the caller pays input tokens for the empty ones. The same applies to the streaming path, which produces the same history items.
OpenRouter documents passing reasoning back on the assistant message, via
message.reasoningormessage.reasoning_details, to preserve reasoning continuity across calls: https://openrouter.ai/docs/guides/best-practices/reasoning-tokensThis is not an OpenAI Chat Completions field, and the SDK already targets it deliberately on both sides.
hasReasoningContent()inopenaiChatCompletionsModel.tsreadsmessage.reasoning, the streaming converter accumulatesdelta.reasoning, andChatCompletionAssistantMessageWithReasoningexists in the converter so it can be written back out. The read path and the write path just do not line up.Fix
When the pending assistant message was created only by a reasoning item, hand its reasoning to the assistant message that immediately follows instead of flushing it as a separate turn:
[ { "role": "user", "content": "What is the capital of France?" }, { "role": "assistant", "content": [ { "type": "text", "text": "The capital of France is **Paris**." } ], "reasoning": "[truncated]" }, { "role": "user", "content": "And Germany?" } ]19 lines in one function. No public API change.
Behavior that is deliberately unchanged, and now covered by tests:
providerData.reasoningon the assistant message still wins, since provider data is spread last.A reasoning item with empty
rawContentno longer produces an assistant message with neither content nor reasoning.Out of scope here, happy to follow up if wanted:
reasoning_detailsis not carried onto the assistant message at all, and the nested copy ofreasoninginside the text content part comes from the existingproviderDatapassthrough inextractAllAssistantContent.Test plan
Six cases in
packages/agents-openai/test/openaiChatCompletionsConverter.test.ts. Four fail onmainand pass with the change; two are guards that pass in both states so the unchanged paths stay pinned:keeps reasoning on the assistant message it belongs todrops an empty reasoning item instead of emitting an empty assistant turndoes not steal reasoning from a previous assistant messagedoes not accumulate contentless assistant messages across turnskeeps reasoning with the tool call message when a tool call follows(guard)does not attach reasoning to a following user message(guard)Commands run locally on Windows:
pnpm build(0 errors)pnpm testforpackages/agents-openai: 21 files, 665 tests passingpnpm test: the only failures are pre-existing on this platform and unrelated to this change (the sandbox suites that require POSIX path separators, and the RunState fixture hash check that trips on CRLF checkouts)pnpm lintpnpm -r build-checkIssue number
N/A
Checks
pnpm testandpnpm test:examplespnpm test:integration(see details).agents/skills/code-change-verification/scripts/run.sh/reviewbefore submitting this PRpnpm changesetto indicate my changes