Skip to content

[BUG] Assistant message text is truncated when reasoning and content are streamed in the same delta #1461

Closed

Description

@daewoongoh

Describe the bug

When using LLM providers (e.g., LiteLLM, Qwen, NanoGPT, etc.), if the API streams both reasoning_content (or equivalent thinking tags) and content within the exact same streaming chunk (Delta), the assistant's message text gets truncated in the UI. This happens because the system processes and commits the text content first, causing the internal task logic to get tangled when reasoning arrives immediately after in the same chunk.

To Reproduce

Steps to reproduce the behavior:

  1. Connect to an API provider/proxy that batches streaming chunks (e.g., LiteLLM) or a model that sends reasoning and content simultaneously.
  2. Trigger a response that contains both a thinking process (reasoning_content) and a final answer (content).
  3. If the network delivers both fields in a single delta object, observe the assistant's response in the chat UI.
  4. See error: The text content gets unexpectedly cut off (truncated) or fails to render properly.

Expected behavior

The system should strictly yield and process the reasoning chunk before the text chunk whenever both are present in the same delta. This ensures the reasoning process is fully rendered and completed before the actual text content begins, preventing any UI truncation.

Screenshots

Video

What version of zoo are you running

Latest version (main branch)

Additional context

  • The issue originates from the for await (const chunk of stream) logic in OpenAI-compatible providers (src/api/providers/*.ts).
  • By default, most providers were yielding delta.content before extractReasoningFromDelta(delta).
  • Reordering the yield sequence to process reasoning first fixes the truncation bug completely. Test cases verifying this exact simultaneous delta scenario have also been written and validated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions