Skip to content

fix(openai-chat): preserve native Mistral thinking on replay - #7814

Merged
lidge-jun merged 2 commits into
lidge-jun:devfrom
adaurban:fix/mistral-native-thinking-replay
Oct 27, 2026
Merged

lidge-jun merged 2 commits into
lidge-jun:devfrom
adaurban:fix/mistral-native-thinking-replay

Conversation

@adaurban

@adaurban adaurban commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Mistral's thinking can appear in Codex on the first turn but disappear from the next model request. The response parser and Responses replay keep the thinking text; the Chat serializer omits it. Adding preserveReasoningContentModels alone sends reasoning_content, rather than Mistral's native thinking chunks.

Replay the prior thinking as native content chunks before assistant text for api.mistral.ai. This applies to hosted GLM and reasoning-enabled Mistral Small, Medium and Large. Explicit preserve lists, including [], retain control.

For example, the second request follows a thinking response and a tool result:

Before After
Assistant text and the tool call are replayed; thinking is missing. The same text and call are replayed with a native thinking chunk.
{"role":"assistant","content":[{"type":"thinking","thinking":[{"type":"text","text":"Synthetic prior thinking."}]},{"type":"text","text":"Checking."}],"tool_calls":[{"id":"call_read","type":"function","function":{"name":"read_file","arguments":"{}"}}]}

Use actual text from the existing scoped replay cache when needed, without inventing placeholders or signatures. The shared prompt projection counts the replayed thinking. Other destinations and native Chat passthrough keep their existing behavior.

This follows #7047's response-parsing fix. Mistral's multi-turn guidance calls for replaying assistant thinking chunks. The fix reconstructs thinking text; it does not restore original chunk boundaries or native signature/closed metadata, which the current parser does not retain.

Verification

Bun 1.4.4, Linux. Base: 736fa5087c0b807cc2c7fd54ebae1548e6b2d4cb (dev, 2.94.0). Published commit: 8414064de8c81dd1922a66eecaeb25e5a7de6262. The branch is two commits behind the publication-time dev tip, f94e95ce5.

  • Before the fix, the 16 new continuation cases fail and all 51 existing cases pass. They cover GLM, Flash, Small and Large with streamed/buffered responses and visible/hidden thinking. Additional cases cover opt-outs, tool-free turns, missing thinking and scoped orphan recovery.
  • bun scripts/test.ts --changed=736fa5087c0b807cc2c7fd54ebae1548e6b2d4cb --parallel=1 tests/adapters/openai/ tests/adapters/translator-budget.test.ts tests/adapters/translator-budget-lifetime.test.ts tests/providers/deepseek-reasoning-replay.test.ts tests/providers/zai-reasoning-replay.test.ts tests/responses/chat-native-reasoning-replay.test.ts: 950 pass across 38 files. The run used upstream/dev while it pointed to the exact base shown here.
  • bun scripts/test.ts --parallel=1 tests/test-layout.test.ts tests/test-layout-tooling.test.ts tests/ci-workflows/structure-ssot.test.ts tests/ci-workflows/repo-hygiene.test.ts tests/ci-workflows/file-size-ratchet.test.ts: 105 pass across five files. These explicit checks include source-as-data guards outside the import graph.
  • bun run typecheck, bun run privacy:scan, bun run structure:check, and git diff --check: pass on the published source.
  • cd docs-site && bun run build: pass with the locked dependencies; 593 pages and 85,001 internal links checked.
  • A separate offline capture harness passes 28 two-turn combinations through the real adapter, Responses bridge and parser, plus six settings and six other-host controls. It also reproduces the omission on unchanged installed 2.93.0, upstream main 2.93.0 and the dev base.

All upstream replies are synthetic. Used focused and import-graph checks to limit local resource use alongside other workstation work; the full suite, cross-platform CI, live model acceptance and external review remain pending. This does not establish a model-quality improvement or a fix for retry loops. Keep the PR in draft until review is complete.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • Required local validation passed; commands, results, and any full-suite exception are documented.

  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

Maintainer note: the checklist was reset only by the maintainer merging dev into this branch (head ed782d9). Contributor validation applies to 8414064; exact-head CI on ed782d9 is green (27/27). Re-ticked by the maintainer for dev integration.

Summary by CodeRabbit

  • New Features
    • For Mistral Chat, supplied reasoning is now replayed as native thinking content before assistant text across hosted model families. An explicit model list controls which models preserve reasoning; an empty list opts out.
  • Bug Fixes
    • Prevented placeholder thinking content from appearing in Mistral responses when cached reasoning is unavailable.
  • Documentation
    • Clarified how Mistral reasoning replay and model-list settings work.

* Replay historical thinking as native content chunks for Mistral-hosted GLM, Small and Large, while honoring explicit opt-outs.
* Keep scoped tool recovery, other destinations and native passthrough unchanged; never invent thinking placeholders or signatures.
* Add two-turn regression coverage and document the text-only replay contract.
@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 8414064 and ed782d9.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between f94e95c and 8414064.

Files selected for processing: 5
  • docs-site/src/content/docs/reference/configuration/providers.md
  • src/adapters/openai-chat/messages.ts
  • src/adapters/openai-chat/wire.ts
  • structure/providers/chat-compat.md
  • tests/adapters/openai/openai-chat-mistral-content.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

The adapter now detects Mistral Chat targets and can replay supplied reasoning as native thinking content before assistant text. Explicit model lists still control enrollment. Tests cover stream and buffered responses, tool-call continuations, cache scope, and opt-outs.

Changes: Mistral thinking replay

Layer / File(s) Summary
Mistral target and replay policy
src/adapters/openai-chat/wire.ts, src/adapters/openai-chat/messages.ts, docs-site/src/content/docs/reference/configuration/providers.md
The adapter identifies api.mistral.ai by hostname. When no explicit preserveReasoningContentModels list is set, Mistral targets preserve supplied reasoning by default. The provider documentation describes this policy.
Native thinking serialization and validation
src/adapters/openai-chat/messages.ts, structure/providers/chat-compat.md, tests/adapters/openai/openai-chat-mistral-content.test.ts
Replayed reasoning is serialized as Mistral thinking content, followed by assistant text when present. Mistral serialization does not add placeholder reasoning when reasoning is absent. Tests cover response modes, opt-outs, and scoped cache recovery. The compatibility documentation describes the reconstructed content and its limits.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant ChatHistory
  participant MessageSerializer
  participant ReplayCache
  participant MistralChatAPI
  ChatHistory->>MessageSerializer: translated assistant history
  MessageSerializer->>ReplayCache: look up scoped tool-call reasoning when needed
  ReplayCache-->>MessageSerializer: cached reasoning text when scope matches
  MessageSerializer->>MistralChatAPI: thinking content followed by assistant text
Loading

Suggested reviewers: lidge-j

Merge Risk: Minimal · previous review 84140

Mistral thinking is replayed as text-only chunks; no actionable merge-blocking defect was established.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within surv… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: preserving native Mistral thinking during OpenAI Chat replay. It matches the files and stated objectives.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 4 functions across 3 files. (2 skipped: 2 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the bug Something isn't working label Oct 27, 2026
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

✅ READY

  • all PR quality gates passed; the review readiness checklist is complete.

Review readiness checklist

  • ✅ Required local validation passed; commands, results, and any full-suite exception are documented.
  • ✅ I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
  • ✅ I resolved all correct Codex and CodeRabbit findings.
  • ✅ My PR is ready for review.

✅ 4/4 boxes ticked.

This pull request has been marked Ready for Review.
The review-ready label marks this PR as ready; review automation runs independently.
Maintainers notified: @lidge-jun @Ingwannu

Hygiene

✅ Deterministic PR hygiene checks passed.

@adaurban

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.

@github-actions
github-actions Bot marked this pull request as ready for review October 27, 2026 08:42
@github-actions
github-actions Bot marked this pull request as draft October 27, 2026 13:18
@lidge-jun

Copy link
Copy Markdown
Owner

Maintainer dev integration decision (per MAINTAINERS.md).

  • Exact head: ed782d97525efe069d795c5fb675a5128be40922 (contributor head 8414064 plus a merge of dev; no rebase).
  • Exact-head CI: all 27 checks on this head completed successfully, including gates and test changed. macOS/Windows jobs are skipped on pull requests by policy.
  • Independent review: PASS with 0 blocking findings. The replayed shape matches Mistral's thinking content chunk; the default applies only when the hostname is exactly api.mistral.ai and preserveReasoningContentModels is unset; an explicit list, including [], stays authoritative; other hosts keep their existing replay fields. Red/green: the turn-two replay tests fail 16/16 on dev and pass 16/16 on the PR.
  • Known limitation: the reconstruction is text-only and does not carry Mistral's optional signature/closed fields.
  • Merge check against current dev (2b5148b12): clean merge tree; the English providers reference stays at 1,995 lines, under the 2,000-line threshold.

The readiness checklist was reset by the maintainer's branch update, so the maintainer marks this ready for integration.

@lidge-jun
lidge-jun marked this pull request as ready for review October 27, 2026 13:40
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-27T13:41:42.267703Z
We couldn’t complete this review. Comment @codex review to try again.
ed782d9 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions
github-actions Bot marked this pull request as draft October 27, 2026 13:40
@github-actions
github-actions Bot marked this pull request as ready for review October 27, 2026 14:01
@lidge-jun
lidge-jun merged commit 962546a into lidge-jun:dev Oct 27, 2026
33 checks passed
@lidge-jun

Copy link
Copy Markdown
Owner

Merged into dev as 962546a from exact head ed782d9 after all checks on that head passed. Thanks @adaurban.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working review-ready

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants