Repository navigation
fix(models): keep escaped newlines inside fences in MindIE native stream - #6478
Merged
WillemJiang merged 1 commit intoOct 8, 2026
Conversation
The no-tool native stream applied _decode_escaped_newlines_outside_fences
to each streamed chunk independently, but a fenced code block almost never
opens and closes within one token-sized chunk, so the per-chunk fence regex
never saw a complete ``` pair and decoded every literal \n inside the fence.
Code like print("a\nb") reached the streamed answer and the persisted
message content with a real newline, while the non-streaming path and the
tool-mode simulated stream preserved it.
Decode the native stream with a fence-stateful decoder instead: it carries
whether the stream is inside a fence across chunks and holds back a
trailing partial ```/`\n` sequence, so the joined stream equals the one-shot
decode of the same reply. A fence left unclosed by a truncated reply keeps
its escapes, matching how an unterminated fence still renders as code.
Document native no-tool streaming in the MindIE decode contract.
WillemJiang
approved these changes
Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
MindIE's no-tool native streaming decodes escaped newlines per chunk, so it rewrites code inside fenced blocks.
MindIEChatModel._astream(backend/packages/harness/deerflow/models/mindie_provider.py:282on currentmain) applies_decode_escaped_newlines_outside_fencesto each streamed chunk independently, but that helper pairs ``` delimiters over its input, and a token-sized chunk almost never contains both delimiters of a fenced block. The pairing therefore never sees the fence, and every literal\nin the chunk — including code inside a fence in the full reply — becomes a real newline. A streamed answer containingreaches the user and the persisted message content with the string escape split across lines, while the non-streaming paths (
_generate/_ageneratevia_patch_result_with_tools, which run the helper once over the full content) and the tool-enabled simulated stream preserve it.backend/docs/CONFIGURATION.mddocuments "Escapes inside fenced code remain unchanged" for synchronous generation, asynchronous generation and tool-enabled simulated streaming; #6334 explicitly left the no-tool native-stream decoder outside its fix, and this closes that gap.What changed
MindIEChatModel._astreamnow decodes with a fence-stateful decoder instead of independent per-chunk calls: it carries whether the stream is inside a ``` fence across the chunks of one completion and reuses_decode_escaped_newlines_outside_fencesfor the text outside fences, so the streaming and non-streaming paths share one implementation and cannot drift again.\(fence marker or escape sequence split across a chunk boundary) is held back until the next chunk decides; text still pending when the stream ends is yielded as one final text chunk, so nothing is dropped.backend/docs/CONFIGURATION.mdnow lists native no-tool streaming in the MindIE decode contract.Surface area
\ninside fenced code blocks, matching the non-streaming and tool-enabled-simulated-stream paths.Bug fix verification
backend/tests/test_mindie_provider.py—TestAStream.test_no_tools_stream_keeps_escaped_newlines_inside_fences,test_no_tools_stream_decode_matches_non_streaming(parametrized over five chunk layouts) andtest_no_tools_stream_flushes_trailing_partial_fences.mainand green on this branch? Yes. On unpatchedmain(a4427625c): 4 failed, 4 passed, 96 deselected. On this branch: 104 passed in the same file._decode_escaped_newlines_outside_fencesover the concatenated raw chunks, covering a complete fence pair in one chunk, fences opening/closing across chunk boundaries, an escape sequence split across a boundary, the ``` delimiter itself split across a boundary, and two fences with prose between them.Validation
All offline with mocked model responses, macOS arm64 / Python 3.12,
uv run --locked:main:PYTHONPATH=. PYTHONIOENCODING=utf-8 PYTHONUTF8=1 uv run --locked pytest tests/test_mindie_provider.py -q --tb=short -k "fence"→ 4 failed, 4 passed, 96 deselected.-k→ 104 passed.MindIEChatModel(model factory, input-sanitization and tool-output-budget middlewares, setup wizard, provider) → 627 passed.make test(four parallel shards) → 25787 passed, 0 failed. An earlier full-suite run hit 17 failures intests/test_web_fetch_relative_links.pycaused by a brokenreadabilipynode_modulesinside this checkout's local.venv(parallel shards raced readabilipy's lazynpm install); the same file passes 48/48 on unpatchedmainand on this branch after repairing the venv, so it is a local environment artifact, not a regression.make lint→ ruff check "All checks passed!", ruff format --check "1906 files already formatted".uv lock --check→ in sync.python scripts/check_agent_guidance.py --base-ref origin/main --head-ref HEAD→ 39 AGENTS.md, 0 errors, 0 warnings.git diff --check→ clean.AI assistance
Tool(s) used: Zcode (Claude-powered autonomous coding agent, submitting as BlueX888).
How you used it: The agent identified the divergence from the #6334 contract, wrote the regression tests and the fix, and ran the validation above. The change is AI-authored; the account owner reviews the diff before merge.