Skip to content

deepseek-v4-flash:cloud: model-leaked literal </think> in assistant history locks agentic clients into a self-sustaining tool-call loop (193 identical calls, ~31M tokens) #17617

Open

Description

@caoer

What is the issue?

deepseek-v4-flash:cloud, driven by Claude Code through the Anthropic-compat endpoint (ANTHROPIC_BASE_URL=http://127.0.0.1:11434, requests visible in the server log as POST /v1/messages?beta=true), degenerated into a self-sustaining loop: 193 consecutive responses, each containing near-identical garbage text plus the exact same tool call, until a human interrupted. Each iteration was a full ~162k-token prompt against Ollama Cloud — roughly 31M input tokens consumed by the loop.

I believe the lock-in mechanism is the one already confirmed in #17248: a literal </think> that reaches message content is reinterpreted as a template delimiter instead of being preserved as text. Here the tag entered through the other door — the model's own output. Once one degenerate response containing a literal </think> landed in the assistant history, the agentic client faithfully replayed it on every subsequent request, the template presumably split the prompt at that literal tag the same way each time, and the model reproduced the same broken continuation indefinitely. The client executes whatever tool call comes back, so the loop feeds itself and nothing model-side ever breaks it.

Timeline (from the client transcript)

  1. Normal operation for ~380 assistant messages. Reasoning arrived correctly as separate thinking blocks throughout (the compat layer's think parsing was working).
  2. At ~162k input tokens, output degraded gradually: a text block of just rath, then inkrath — ordinary long-context repetition wobble.
  3. Then the loop locked in. Every response became: one short garbage text block + the identical Bash tool_use (cd <repo> && grep -n "rebuild" crates/policy/src/defs/rebuild.rs | head -8). 193 iterations.
  4. Only a human interrupt ended it. After the interrupt injected a new user message, the same session recovered completely — normal thinking blocks and correct behavior up to 259k input tokens.

The leaked tags

The garbage text blocks are fragments of a short repeating token cycle, and many contain a raw, unbalanced </think> — a literal tag delivered in the text channel of /v1/messages responses. Distinct variants and counts across the 193 loop responses:

count literal text block content
88 ink responserath
44 ink</think>rath
31 rath
31 ink responserathink response
8 rathink response
5 rathink</think>rath
5 ink</think>
3 ink response
2 rathink responserath

A literal </think> should never be able to reach the text channel unescaped — and per #17248, once it is replayed back in as content, prompt processing splits on it. The combination turns a one-off sampling wobble into an unbreakable loop for any agentic client.

Secondary symptom: usage reporting froze

Every response during the loop reported exactly "input_tokens": 162214, "output_tokens": 0 — frozen across all 193 calls, despite each response containing text and a tool call, and despite the history growing each iteration. Before and after the loop, usage numbers varied normally. This looks like the usage plumbing on the compat path also breaks while the stream is in this state (possibly related to #17130).

Why this deserves attention beyond #17248

#17248 demonstrates the corruption with a literal </think> deliberately placed in user content. This report shows the tag can be injected by the model itself under long-context degeneration, after which the template-splitting behavior makes the failure self-sustaining and expensive: agentic clients (Claude Code, and presumably any tool-loop client) will replay the corrupted history forever. Cost here was ~31M input tokens for a single stuck session; nothing server-side or client-side ever terminates it.

Suggested fixes, in order of leverage:

  1. Escape or strip think-delimiter literals when rendering message content into the prompt (the Cloud API corrupts literal </think> in user messages across thinking models #17248 fix) — this breaks the feedback loop even if the model wobbles.
  2. Never emit a literal </think> in the text channel of the compat APIs — if tag parsing desyncs mid-stream, drop or escape the delimiter rather than passing it through.

Related: #17248 (literal </think> corrupted in message content, same model among the repros), #16375 (raw DSML markup leaking into output on deepseek-v4-flash:cloud — same special-token handling class).

Relevant log output

[GIN] 2026/08/07 - 23:29:59 | 200 | 12.157298167s | 127.0.0.1 | POST "/v1/messages?beta=true"

(server log shows only the routine 200s; all evidence above is from the client-side transcript of the /v1/messages responses)

OS

macOS (darwin-arm64)

GPU

Apple

CPU

Apple

Ollama version

server 0.32.6 (client 0.32.4)

Activity

  1. Saad-Mallebhari commented on Aug 8, 2026

    @Saad-Mallebhari

    Hi @caoer,

    I investigated the root cause of this feedback loop in Ollama's prompt renderer.

    In model/renderers/deepseek3.go (lines 140–145):

    if strings.Contains(content, "</think>") {
        parts := strings.SplitN(content, "</think>", 2)
        if len(parts) > 1 {
            content = parts[1]
        }
    }

    When assistant content contains a literal </think> without a preceding <think> tag (e.g., in inkrath), SplitN unconditionally discards parts[0] ("ink"), truncating the assistant message history down to parts[1] ("rath").

    On subsequent turns, the prompt context for the assistant turn is rendered missing its prior message history. The model loses context, reproduces inkrath, and gets locked into the self-sustaining loop.

    We can fix this by updating deepseek3.go (and related thinking renderers) to ensure content is only trimmed if a preceding <think> tag exists (strings.Index(content, "<think>") < strings.Index(content, "</think>")).

    I can open a PR with this fix and a unit test if helpful!

  2. rick-github commented on Aug 8, 2026

    @rick-github
    Collaborator

    In model/renderers/deepseek3.go (lines 140–145):

    The issue is with the cloud service, the local render has nothing to do with it.

  3. gokcanakalin commented on Aug 17, 2026

    @gokcanakalin

    Is there any development with this issue? I have modified my tools to remove "< think >" and "< /think >" to replace them with [THINK-START] and [THINK-END] . However, I dont understand why should we (as subscribers of ollama cloud) have to modify every agentic coding tool (that we use) to fix this issue? For some unlucky reason, if this happens, the ollama cloud model basically goes off the rails usually. This is the first time, I see such a weird bug that basically corrupts the whole session (if it happens due to reading some material that contains these strings). In such a case, you basically need to scan all the materials you work with and eliminate these strings, which can happen easily if it is stored as a memory or saved into a file or something similar in the project folder that you are working on.

    Here (you can easily confirm this):
    Image

  4. LAN-TINA-WS commented on Aug 20, 2026

    @LAN-TINA-WS

    Same here with deepseek-v4-flash:0731. For me it's the thinking that loops (same reasoning block 221 times, ~1m45s, then failed) rather than tool calls. Same task runs fine on the official DeepSeek API.

  5. jeong-sik commented on Sep 5, 2026

    @jeong-sik

    Following up on the renderer thread above, since it otherwise reads as unresolved.

    @rick-github is right: this issue is the cloud service. <model>:cloud requests are proxied upstream in server/routes.go (req.Model = m.Config.RemoteModel) and never reach model/renderers/deepseek3.go, and #17248 shows the same literal-tag corruption in user messages across several model families. Nothing in the local renderer explains what is reported here, so this should stay open.

    The truncation @Saad-Mallebhari found is real on the local path, so I sent it separately as #18260 (a stray </think>, or content in front of a real <think> pair, no longer cuts the head off replayed assistant history). That PR does not change the cloud behaviour described here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions