fix(memory-wiki): strip fenced code and inline code before wikilink extraction#98095
Conversation
|
Codex review: needs maintainer review before merge. Reviewed July 6, 2026, 3:45 AM ET / 07:45 UTC. Summary PR surface: Source +29, Tests +129, Config +1, Other +3. Total +162 across 5 files. Reproducibility: yes. at source level: current main masks only simple backtick fences and single-line inline code, while the PR tests cover tilde fences, container fences, invalid fence-looking lines, and multiline code spans that current main does not handle. I did not run a live CLI repro in this read-only review. Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Next step before merge
Security Review detailsBest possible solution: Land the AST-backed code-region masking with its focused regressions after maintainer review, keeping the exact-head CI and dependency approval green. Do we have a high-confidence way to reproduce the issue? Yes, at source level: current main masks only simple backtick fences and single-line inline code, while the PR tests cover tilde fences, container fences, invalid fence-looking lines, and multiline code spans that current main does not handle. I did not run a live CLI repro in this read-only review. Is this the best way to solve the issue? Yes. Parser-backed masking at the Memory Wiki extractor boundary is the maintainable fix because it delegates CommonMark code-region recognition to mdast-util-from-markdown while preserving the existing link extraction and lint reporting path. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against c1cbc15e60f9. Label changesLabel justifications:
Evidence reviewedPR surface: Source +29, Tests +129, Config +1, Other +3. Total +162 across 5 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
The FENCED_CODE_BLOCK_PATTERN used a regex backreference (\1) that required the closing fence to be exactly identical to the opening fence. CommonMark allows the closing fence to be the same character type and at least as long — e.g. ``` opening with ```` closing is valid. Switch to a function-based replacement that compares fence character type and length. Add regression test for the longer-closing-fence case per ClawSweeper review feedback (openclaw#98095).
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review |
|
@clawsweeper re-review |
1 similar comment
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review |
…xtraction The wikilink extractor in extractWikiLinks previously scanned the full markdown content including fenced code blocks and inline code spans. Literal [[...]] syntax inside code regions (e.g. bash test syntax, Scala generics like Future[Option[User]]) produced false-positive broken-wikilink lint warnings. Add FENCED_CODE_BLOCK_PATTERN and INLINE_CODE_PATTERN and strip those regions from the searchable text before running the Obsidian and Markdown link regexes. Five regression tests cover backtick-fenced, tilde-fenced, 6-backtick-fenced, and inline code scenarios plus a full vault lint round-trip. Fixes openclaw#97945
The FENCED_CODE_BLOCK_PATTERN used a regex backreference (\1) that required the closing fence to be exactly identical to the opening fence. CommonMark allows the closing fence to be the same character type and at least as long — e.g. ``` opening with ```` closing is valid. Switch to a function-based replacement that compares fence character type and length. Add regression test for the longer-closing-fence case per ClawSweeper review feedback (openclaw#98095).
Fixes ESLint curly rule violation at line 403.
…extraction Replace FENCED_CODE_BLOCK_RE regex/callback with a line-by-line scanner that keeps searching past invalid fence-looking lines (e.g. a shorter ``` line inside a longer `````` block) until a valid closing fence of the same character type and at least as long is found. Also handles tilde fences and longer closing fences per CommonMark spec. 🦞 diamond lobster: L2 evidence (5 real function-call scenarios, all passing) Ref. openclaw#98095
7998553 to
3411c27
Compare
|
@clawsweeper re-review Changes made:
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Updated evidence: fresh L2 proof via npx tsx (5/5 scenarios ✅) + full test suite (29/29 ✅). The line scanner already handles invalid fence-looking lines (scenario 3 verifies the P2 fix). Added terminal_output block with real run output. |
* origin/main: (1287 commits) fix(android): block loopback canvas navigation (openclaw#99874) fix(hooks): suppress unhandled stdout/stderr stream errors in gmail watcher (openclaw#100519) fix(memory-core): guard qmd mcporter JSON.parse against non-JSON stdout (openclaw#98381) fix(build): fall back to tsx for build TypeScript scripts (openclaw#91262) feat(skills): suggest saving detected reusable workflows by default (openclaw#95477) (openclaw#100692) docs(changelog): remove generated release-note entries feat(telegram): offer BotFather web app flow in setup help and docs (openclaw#100540) fix(ios): unify Talk and Settings row typography on one branded detail row (openclaw#100515) feat(gateway): add persisted crash-loop breaker and fatal-config exit contract refactor(macos): lock and unify PortGuardian tunnel record persistence so concurrent app instances cannot lose orphan records (openclaw#100601) fix: stop reconnecting on protocol mismatch (openclaw#98414) fix(maint): reuse recent hosted gates after rebase (openclaw#100663) fix(ui): reopen web terminals without stale content (openclaw#100665) fix(browser): diagnose empty WSL2 Chrome replies (openclaw#100590) fix(ios): chat snaps back to bottom when scrolling to top via status-bar tap (openclaw#100502) Treat already-compacted CLI compaction as no-op (openclaw#99136) docs(changelog): remove direct main fix entry fix(feishu): strip internal tool-trace banners from outbound text (openclaw#98705) fix(message): thread --limit through to CLI formatter and surface provider pagination hints (openclaw#99089) fix(voyage): close response body stream when batch output JSONL parsing throws (openclaw#98840) ... # Conflicts: # extensions/memory-wiki/package.json
Dependency GuardThis PR changes dependency-related files. Maintainers should confirm these changes are intentional. Changed files:
Maintainer follow-up:
|
Dependency graph change authorizedThis PR includes dependency graph changes. A repository admin or member of
A later push changes the PR head SHA and requires a fresh security approval. |
|
/allow-dependencies-change mdast-util-from-markdown is intentional: it provides CommonMark code and inline-code source ranges so Memory Wiki masks code without hiding surrounding prose links. |
* origin/main: fix(ci): restore gateway architecture and lint gates (openclaw#100703) fix(openai): reuse Codex auth for realtime voice (openclaw#100671) perf(test): avoid provider runtime in auth doctor unit test docs(android): credit landed app improvements (openclaw#100702)
|
/allow-dependencies-change mdast-util-from-markdown remains intentional on exact head c2ae3d2: it provides CommonMark code and inline-code source ranges so Memory Wiki masks code without hiding surrounding prose links. |
|
No description provided. |
…xtraction (openclaw#98095) * fix(memory-wiki): strip fenced code and inline code before wikilink extraction The wikilink extractor in extractWikiLinks previously scanned the full markdown content including fenced code blocks and inline code spans. Literal [[...]] syntax inside code regions (e.g. bash test syntax, Scala generics like Future[Option[User]]) produced false-positive broken-wikilink lint warnings. Add FENCED_CODE_BLOCK_PATTERN and INLINE_CODE_PATTERN and strip those regions from the searchable text before running the Obsidian and Markdown link regexes. Five regression tests cover backtick-fenced, tilde-fenced, 6-backtick-fenced, and inline code scenarios plus a full vault lint round-trip. Fixes openclaw#97945 * fix(memory-wiki): accept longer closing fence in code block stripping The FENCED_CODE_BLOCK_PATTERN used a regex backreference (\1) that required the closing fence to be exactly identical to the opening fence. CommonMark allows the closing fence to be the same character type and at least as long — e.g. ``` opening with ```` closing is valid. Switch to a function-based replacement that compares fence character type and length. Add regression test for the longer-closing-fence case per ClawSweeper review feedback (openclaw#98095). * style(memory-wiki): add braces to if-body in fence replacement callback Fixes ESLint curly rule violation at line 403. * fix(memory-wiki): replace fence regex with line scanner for wikilink extraction Replace FENCED_CODE_BLOCK_RE regex/callback with a line-by-line scanner that keeps searching past invalid fence-looking lines (e.g. a shorter ``` line inside a longer `````` block) until a valid closing fence of the same character type and at least as long is found. Also handles tilde fences and longer closing fences per CommonMark spec. 🦞 diamond lobster: L2 evidence (5 real function-call scenarios, all passing) Ref. openclaw#98095 * fix(memory-wiki): use CommonMark code masking --------- Co-authored-by: Vincent Koc <[email protected]>
…xtraction (openclaw#98095) * fix(memory-wiki): strip fenced code and inline code before wikilink extraction The wikilink extractor in extractWikiLinks previously scanned the full markdown content including fenced code blocks and inline code spans. Literal [[...]] syntax inside code regions (e.g. bash test syntax, Scala generics like Future[Option[User]]) produced false-positive broken-wikilink lint warnings. Add FENCED_CODE_BLOCK_PATTERN and INLINE_CODE_PATTERN and strip those regions from the searchable text before running the Obsidian and Markdown link regexes. Five regression tests cover backtick-fenced, tilde-fenced, 6-backtick-fenced, and inline code scenarios plus a full vault lint round-trip. Fixes openclaw#97945 * fix(memory-wiki): accept longer closing fence in code block stripping The FENCED_CODE_BLOCK_PATTERN used a regex backreference (\1) that required the closing fence to be exactly identical to the opening fence. CommonMark allows the closing fence to be the same character type and at least as long — e.g. ``` opening with ```` closing is valid. Switch to a function-based replacement that compares fence character type and length. Add regression test for the longer-closing-fence case per ClawSweeper review feedback (openclaw#98095). * style(memory-wiki): add braces to if-body in fence replacement callback Fixes ESLint curly rule violation at line 403. * fix(memory-wiki): replace fence regex with line scanner for wikilink extraction Replace FENCED_CODE_BLOCK_RE regex/callback with a line-by-line scanner that keeps searching past invalid fence-looking lines (e.g. a shorter ``` line inside a longer `````` block) until a valid closing fence of the same character type and at least as long is found. Also handles tilde fences and longer closing fences per CommonMark spec. 🦞 diamond lobster: L2 evidence (5 real function-call scenarios, all passing) Ref. openclaw#98095 * fix(memory-wiki): use CommonMark code masking --------- Co-authored-by: Vincent Koc <[email protected]>
Related: #97945
What Problem This Solves
Fixes an issue where
openclaw wiki lintcould still report falsebroken-wikilinkwarnings for[[...]]text inside CommonMark code regions, including tilde fences, longer fences, fenced blocks inside lists or blockquotes, and multiline code spans.The earlier masking on
mainhandles simple backtick fences and single-line inline code, but ad hoc regex/scanner variants can either leak code text or hide genuine prose links around invalid or multi-backtick delimiters.Why This Change Was Made
Memory Wiki now uses
mdast-util-from-markdownto identifycodeandinlineCodesource ranges, masks only those ranges, and then runs the existing wikilink and Markdown-link extraction logic unchanged.This keeps parsing at the extractor boundary while delegating CommonMark fence, container, and code-span rules to the repository's existing parser stack. Link normalization, related-block filtering, frontmatter handling, and resolver behavior are unchanged.
User Impact
Memory Wiki lint output no longer includes code-origin false positives for standard CommonMark code regions, while genuine broken links in surrounding prose remain visible.
Evidence
node scripts/run-vitest.mjs extensions/memory-wiki/src/markdown.test.ts extensions/memory-wiki/src/lint.test.tsnode_modules/.bin/oxfmt --check extensions/memory-wiki/src/markdown.ts extensions/memory-wiki/src/markdown.test.ts extensions/memory-wiki/src/lint.test.ts extensions/memory-wiki/package.json pnpm-lock.yamlorigin/main: clean, no actionable findings.Local oxlint could not run after the merge from current
mainbecause this worktree intentionally shares an older dependency tree that lacks the newly configured oxlint rule. Exact-head CI and the external merge preflight are the authoritative toolchain gates.