fix(markdown): a fenced-code line with trailing text is content, not a closing fence#96745
Merged
vincentkoc merged 2 commits intoJun 26, 2026
Merged
Conversation
Contributor
|
Codex review: needs maintainer review before merge. Reviewed June 26, 2026, 5:27 PM ET / 21:27 UTC. Summary PR surface: Source +4, Tests +45. Total +49 across 2 files. Reproducibility: yes. Source inspection on current main shows the close branch ignores trailing text after a matching marker, and the PR body includes before/after exported-function output for the failing parser case. Review metrics: none identified. Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Next step before merge
Security Review detailsBest possible solution: Land this canonical shared-scanner fix after maintainer review, then close the duplicate salvage PR as redundant. Do we have a high-confidence way to reproduce the issue? Yes. Source inspection on current main shows the close branch ignores trailing text after a matching marker, and the PR body includes before/after exported-function output for the failing parser case. Is this the best way to solve the issue? Yes. Fixing scanFenceSpans is the best owner-boundary fix because the downstream chunkers and scanners derive fence safety from that shared parser. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 29680046801d. Label changesLabel justifications:
Evidence reviewedPR surface: Source +4, Tests +45. Total +49 across 2 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
clawsweeper
Bot
added
proof: sufficient
…a closing fence scanFenceSpans accepted any line starting with >=3 matching fence markers as a closing fence, ignoring trailing text after the marker. Per CommonMark a closing fence may be followed only by whitespace, so a code-content line such as "``` not a close" was wrongly treated as a close: the block ended early, the following lines were reported as outside any fence, and the trailing marker line became a new unclosed opener. That made isSafeFenceBreak() return true for offsets inside the real code block and findFenceSpanAt() return undefined, so chunkers (chunkMarkdownText, the embedded-agent block chunker) could split inside a fenced code block — the exact thing this module exists to prevent. Require the closing fence's trailing text to be whitespace-only. Opening info strings, bare closes, and longer same-marker closes are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Co-authored-by: ly-wang19 <[email protected]>
vincentkoc
force-pushed
the
fix/fence-close-trailing-text
branch
from
June 26, 2026 21:22
f63d668 to
7104845
Compare
Member
|
Merged via squash.
|
wangmiao0668000666
pushed a commit
to wangmiao0668000666/openclaw
that referenced
this pull request
Jun 27, 2026
…a closing fence (openclaw#96745) * fix(markdown): a fenced-code line with trailing text is content, not a closing fence scanFenceSpans accepted any line starting with >=3 matching fence markers as a closing fence, ignoring trailing text after the marker. Per CommonMark a closing fence may be followed only by whitespace, so a code-content line such as "``` not a close" was wrongly treated as a close: the block ended early, the following lines were reported as outside any fence, and the trailing marker line became a new unclosed opener. That made isSafeFenceBreak() return true for offsets inside the real code block and findFenceSpanAt() return undefined, so chunkers (chunkMarkdownText, the embedded-agent block chunker) could split inside a fenced code block — the exact thing this module exists to prevent. Require the closing fence's trailing text to be whitespace-only. Opening info strings, bare closes, and longer same-marker closes are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(markdown): honor fence suffix whitespace rules Co-authored-by: ly-wang19 <[email protected]> --------- Co-authored-by: ly-wang19 <[email protected]> Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> Co-authored-by: Vincent Koc <[email protected]>
github-actions Bot
pushed a commit
to Desicool/openclaw
that referenced
this pull request
Jun 27, 2026
…a closing fence (openclaw#96745) * fix(markdown): a fenced-code line with trailing text is content, not a closing fence scanFenceSpans accepted any line starting with >=3 matching fence markers as a closing fence, ignoring trailing text after the marker. Per CommonMark a closing fence may be followed only by whitespace, so a code-content line such as "``` not a close" was wrongly treated as a close: the block ended early, the following lines were reported as outside any fence, and the trailing marker line became a new unclosed opener. That made isSafeFenceBreak() return true for offsets inside the real code block and findFenceSpanAt() return undefined, so chunkers (chunkMarkdownText, the embedded-agent block chunker) could split inside a fenced code block — the exact thing this module exists to prevent. Require the closing fence's trailing text to be whitespace-only. Opening info strings, bare closes, and longer same-marker closes are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(markdown): honor fence suffix whitespace rules Co-authored-by: ly-wang19 <[email protected]> --------- Co-authored-by: ly-wang19 <[email protected]> Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> Co-authored-by: Vincent Koc <[email protected]>
xydigit-zt
pushed a commit
to xydigit-zt/xydigit-zt-openclaw
that referenced
this pull request
Jun 28, 2026
…a closing fence (openclaw#96745) * fix(markdown): a fenced-code line with trailing text is content, not a closing fence scanFenceSpans accepted any line starting with >=3 matching fence markers as a closing fence, ignoring trailing text after the marker. Per CommonMark a closing fence may be followed only by whitespace, so a code-content line such as "``` not a close" was wrongly treated as a close: the block ended early, the following lines were reported as outside any fence, and the trailing marker line became a new unclosed opener. That made isSafeFenceBreak() return true for offsets inside the real code block and findFenceSpanAt() return undefined, so chunkers (chunkMarkdownText, the embedded-agent block chunker) could split inside a fenced code block — the exact thing this module exists to prevent. Require the closing fence's trailing text to be whitespace-only. Opening info strings, bare closes, and longer same-marker closes are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(markdown): honor fence suffix whitespace rules Co-authored-by: ly-wang19 <[email protected]> --------- Co-authored-by: ly-wang19 <[email protected]> Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> Co-authored-by: Vincent Koc <[email protected]>
QiuYuang
pushed a commit
to QiuYuang/openclaw
that referenced
this pull request
Jul 1, 2026
…a closing fence (openclaw#96745) * fix(markdown): a fenced-code line with trailing text is content, not a closing fence scanFenceSpans accepted any line starting with >=3 matching fence markers as a closing fence, ignoring trailing text after the marker. Per CommonMark a closing fence may be followed only by whitespace, so a code-content line such as "``` not a close" was wrongly treated as a close: the block ended early, the following lines were reported as outside any fence, and the trailing marker line became a new unclosed opener. That made isSafeFenceBreak() return true for offsets inside the real code block and findFenceSpanAt() return undefined, so chunkers (chunkMarkdownText, the embedded-agent block chunker) could split inside a fenced code block — the exact thing this module exists to prevent. Require the closing fence's trailing text to be whitespace-only. Opening info strings, bare closes, and longer same-marker closes are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(markdown): honor fence suffix whitespace rules Co-authored-by: ly-wang19 <[email protected]> --------- Co-authored-by: ly-wang19 <[email protected]> Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> Co-authored-by: Vincent Koc <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Problem This Solves
scanFenceSpans/parseFenceSpans(packages/markdown-core/src/fences.ts) maps where fenced code blocks start and end so chunkers never split inside one. It accepted any line that begins with ≥3 matching fence markers as a closing fence, ignoring trailing text after the marker. Per CommonMark a closing fence may be followed only by whitespace, so a code-content line like``` not a closewas wrongly treated as a close: the block ended early, the following lines were reported as outside any fence, and the trailing real```became a new unclosed opener.The consequence:
isSafeFenceBreak()returnstruefor offsets inside the actual code block andfindFenceSpanAt()returnsundefined, sochunkMarkdownTextand the embedded-agent block chunker can split inside a fenced code block — the exact thing this module exists to prevent. Code blocks that demonstrate markdown/fences (a line starting with ``` followed by text) are ordinary content.Why This Change Was Made
The closing branch already checked the marker character and length (its comment cites CommonMark) but missed the trailing-text rule. The fix adds that one condition: a closing fence's trailing text must be whitespace-only. The reference implementation (micromark) confirms the sample input is a single code block. This makes fence detection stricter — the safe direction (it keeps more text inside a fence, so a chunker is less likely to split inside code).
User Impact
Outbound/agent text containing a fenced code block whose body has a
```-prefixed line is no longer mis-parsed, so chunkers won't break the code block at that point. Opening info strings, bare closes, and longer same-marker closes are unaffected. No API change.Evidence
oxfmt,oxlint,tsgo:corepass. A new colocatedfences.test.tsregression test passes and fails when the fix is reverted (fail-before). Both downstream suites —src/auto-reply/chunk.test.tsandsrc/agents/embedded-agent-block-chunker.test.ts— stay green.Real behavior proof
Behavior addressed: a fenced-code content line beginning with the fence marker but carrying trailing text (
``` not a close) was treated as a closing fence, soisSafeFenceBreakreported an offset inside the code block as a safe chunk break, letting chunkers split inside the block.Real environment tested: Ran the exported
parseFenceSpans/isSafeFenceBreak/findFenceSpanAtfrompackages/markdown-core/src/fences.tsviatsxon Node v22.22.1 (no mocks), before and after the patch, plus the colocated Vitest test and the two downstream chunker suites vianode scripts/run-vitest.mjs.Exact steps or command run after this patch:
parseFenceSpans("```\\ncode\\n``` not a close\\nmore code\\n```\\n")andisSafeFenceBreak(spans, <offset of "more code">); thennode scripts/run-vitest.mjs packages/markdown-core/src/fences.test.ts src/auto-reply/chunk.test.ts src/agents/embedded-agent-block-chunker.test.ts,node scripts/run-oxlint.mjs,pnpm tsgo:core.Evidence after fix:
Observed result after fix: a marker line with trailing text stays inside the code block (one span), interior offsets are no longer reported as safe breaks, and valid closes (bare, longer same-marker, with trailing whitespace) plus opening info strings still behave correctly; the downstream chunkers are unchanged on their existing tests.
What was not tested: a full end-to-end channel render of a chunked message containing such a code block (no live channel in this environment); the fix is a pure parser change proven at the function and unit-test level, cross-checked against the CommonMark reference (micromark) for the single-code-block interpretation.