Skip to content

fix(memory-wiki): tolerate YAML parse errors in wiki frontmatter#96130

Closed
hanZeng-08 wants to merge 1 commit into
openclaw:mainfrom
hanZeng-08:fix/96125-wiki-yaml-parse-error-tolerance
Closed

fix(memory-wiki): tolerate YAML parse errors in wiki frontmatter#96130
hanZeng-08 wants to merge 1 commit into
openclaw:mainfrom
hanZeng-08:fix/96125-wiki-yaml-parse-error-tolerance

Conversation

@hanZeng-08

@hanZeng-08 hanZeng-08 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Fixes #96125

What Problem This Solves

Fixes an issue where a single wiki page with malformed YAML frontmatter causes wiki_apply and wiki_lint to fail entirely, even though the file was already written successfully. The compileMemoryWikiVault step throws YAMLParseError ("Unexpected scalar at node end") when parsing frontmatter containing markdown bold syntax with colons (e.g., - **MEMORY.md 第 235 行**:"..."). This creates a single-point-of-failure: one bad file breaks the entire vault.

Why This Change Was Made

parseWikiMarkdown (extensions/memory-wiki/src/markdown.ts:176) called YAML.parse without any error handling. When a list item in frontmatter contains markdown bold + colon (which YAML interprets as an invalid key-value structure), the parser throws and the error propagates up through toWikiPageSummary → readPageSummaries → compileMemoryWikiVault → wiki_apply/wiki_lint.

The fix wraps YAML.parse in a try-catch: when parsing fails, return an empty frontmatter but preserve the body. This prevents one bad file from crashing the entire vault compilation, while allowing the page to still be processed (albeit without frontmatter metadata).

User Impact

Users with wiki pages containing complex markdown in frontmatter (like bold + colon combinations) will no longer see wiki_apply/wiki_lint crash. The affected page will be processed without its frontmatter metadata, but the rest of the vault compiles normally. Previously, the entire operation would fail with a cryptic YAML parse error.

Evidence

Environment: Linux, Node v22.23.0, pnpm v11.2.2, worktree SHA 2af0604

Before fix (reproduction with yaml package):

$ node -e "const YAML = require('yaml'); YAML.parse('- **MEMORY.md 第 235 行**:\"加特契纳...\"')"
YAMLParseError: Unexpected scalar at node end at line 1, column 10:

- **MEMORY.md 第 235 行**: "加特契纳..."
         ^

After fix (new regression test on new code):

$ pnpm test extensions/memory-wiki/src/markdown.test.ts
Test Files  1 passed (1)
     Tests   8 passed (8)

Lint/format clean:

$ npx oxlint@latest extensions/memory-wiki/src/markdown.ts extensions/memory-wiki/src/markdown.test.ts
Found 0 warnings and 0 errors.
$ pnpm format extensions/memory-wiki/src/markdown.ts extensions/memory-wiki/src/markdown.test.ts
Finished in 13ms on 2 files using 1 threads.

@hanZeng-08
fix(memory-wiki): tolerate YAML parse errors in wiki frontmatter
4f66821 
A single wiki page with malformed YAML frontmatter (e.g. markdown bold
syntax containing colons inside list items) caused wiki_apply and
wiki_lint to fail entirely via YAMLParseError.

parseWikiMarkdown now wraps YAML.parse in try-catch: when parsing fails,
it returns an empty frontmatter while preserving the body. This prevents
one bad file from crashing the entire vault compilation.

Fixes openclaw#96125
@hanZeng-08

hanZeng-08 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Fix rationale

Problem

A single wiki page with malformed YAML frontmatter (e.g., - **MEMORY.md 第 235 行**:"...") crashes wiki_applyandwiki_lintwithYAMLParseError. The error propagates through parseWikiMarkdown→toWikiPageSummary→readPageSummaries→compileMemoryWikiVault`, stopping the entire vault compilation.

Why this approach

Option A: fix the YAML content — rejected. The content is user-authored markdown; we can't control what users put in frontmatter. Requiring users to avoid colons inside bold text is unrealistic.

Option B: change readPageSummaries errorMode from "stop" to "continue" — rejected. This would skip the entire page (both frontmatter and body), causing data loss. The page would disappear from the wiki entirely.

Option C: wrap YAML.parse in try-catch, return empty frontmatter on failure — adopted. This is the minimal fix:

  • One bad file doesn't crash the vault
  • The page still compiles (body is preserved)
  • Other pages are unaffected
  • No schema, config, or API changes
  • Backward compatible: valid YAML works exactly as before

Trade-off

A page with invalid YAML frontmatter loses its metadata (title, sourceIds, claims, etc.) and falls back to body-only rendering. This is acceptable because:

  1. The page still exists in the wiki
  2. The user can fix the frontmatter later
  3. The alternative is the entire vault failing

Safety

  • No changes to renderWikiMarkdown (writing path unchanged)
  • No changes to vault layout, index format, or search
  • Only parseWikiMarkdown (reading path) affected
  • Existing tests pass; new regression test covers the failure case

@hanZeng-08

hanZeng-08 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot mentioned this pull request Jun 24, 2026
Closed
@clawsweeper

clawsweeper Bot commented Jun 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 23, 2026, 8:40 PM ET / 00:40 UTC.

Summary
The PR wraps memory-wiki YAML frontmatter parsing in parseWikiMarkdown with a catch that returns empty frontmatter while preserving the markdown body, plus a regression test for malformed frontmatter.

PR surface: Source +4, Tests +12. Total +16 across 2 files.

Reproducibility: yes. Source inspection shows malformed frontmatter reaches parseWikiMarkdown, YAML.parse throws, and readPageSummaries rethrows through vault compilation; I did not run a live tool repro because this review is read-only.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: vector/embedding metadata: extensions/memory-wiki/src/markdown.test.ts, vector/embedding metadata: extensions/memory-wiki/src/markdown.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #96125
Summary: This PR and #96177 are competing candidate fixes for the same canonical memory-wiki YAML frontmatter bug.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦪 silver shellfish
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Move the catch out of parseWikiMarkdown and into vault-wide summary/lint handling so write-target parsing still fails safely.
  • [P1] Add regression coverage proving wiki_apply does not rewrite a malformed target page with empty frontmatter.
  • [P1] Add redacted terminal output or logs showing wiki_apply and wiki_lint after the fix on a real vault containing a malformed unrelated page.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR provides a YAML package snippet and unit-test output, but needs redacted terminal output or logs showing after-fix wiki_apply and wiki_lint behavior on a real vault; updating the PR body should trigger a fresh ClawSweeper review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The global parser catch can turn invalid edit-target frontmatter into an empty object; later wiki_apply writes can then replace existing metadata that should have remained protected by a parse failure.
  • [P1] The PR body shows a dependency reproduction and unit-test output, but no after-fix wiki_apply or wiki_lint run against a real vault with a malformed unrelated page.

Maintainer options:

  1. Move Tolerance To Summary Reads (recommended)
    Keep direct parser callers strict, catch YAML failures in toWikiPageSummary or the vault scan, and surface a dedicated lint issue for the offending page.
  2. Choose The Sibling Fix After Proof
    Maintainers can prefer fix(memory-wiki): wiki_apply errors when another page has bad frontmatter #96177 if it gets adequate real behavior proof and remains the cleaner implementation path.
  3. Accept Silent Metadata Loss Explicitly
    If maintainers intentionally want all parser callers to degrade to empty frontmatter, the PR needs a visible data-loss rationale and upgrade proof before merge.

Next step before merge

  • [P1] Maintainers need to choose between reworking this PR and the sibling candidate, and the contributor still needs real behavior proof; this is not safe for automated repair markers.

Security
Cleared: The diff only changes memory-wiki TypeScript and tests; it does not add dependencies, workflows, secrets handling, install scripts, or package metadata.

Review findings

  • [P1] Keep parser failures strict for write targets — extensions/memory-wiki/src/markdown.ts:193
Review details

Best possible solution:

Keep parseWikiMarkdown strict for edit-target reads, add tolerance in the vault-wide summary/lint path, and prove wiki_apply plus wiki_lint on a real vault that contains one malformed unrelated page.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows malformed frontmatter reaches parseWikiMarkdown, YAML.parse throws, and readPageSummaries rethrows through vault compilation; I did not run a live tool repro because this review is read-only.

Is this the best way to solve the issue?

No. The PR fixes the crash in one path, but the best fix is to catch malformed frontmatter only while summarizing unrelated vault pages and keep direct edit-target parsing strict.

Full review comments:

  • [P1] Keep parser failures strict for write targets — extensions/memory-wiki/src/markdown.ts:193
    This catch changes parseWikiMarkdown for every caller, but wiki_apply uses that helper to parse pages it is about to rewrite. If an existing target page has invalid frontmatter, the new {} fallback flows into the write path and can replace existing metadata instead of stopping the edit; confine the tolerance to vault-wide summary/lint reads and keep direct edit-target parsing strict.
    Confidence: 0.91

Overall correctness: patch is incorrect
Overall confidence: 0.9

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against cd7e3df1eac9.

Label changes

Label changes:

  • add P2: This fixes a concrete memory-wiki regression with limited blast radius, but the current patch has a merge-blocking correctness issue.
  • add merge-risk: 🚨 compatibility: Merging this patch can change existing malformed wiki pages from fail-closed parsing to rewrite-with-empty-metadata behavior during later edits.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR provides a YAML package snippet and unit-test output, but needs redacted terminal output or logs showing after-fix wiki_apply and wiki_lint behavior on a real vault; updating the PR body should trigger a fresh ClawSweeper review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: This fixes a concrete memory-wiki regression with limited blast radius, but the current patch has a merge-blocking correctness issue.
  • merge-risk: 🚨 compatibility: Merging this patch can change existing malformed wiki pages from fail-closed parsing to rewrite-with-empty-metadata behavior during later edits.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR provides a YAML package snippet and unit-test output, but needs redacted terminal output or logs showing after-fix wiki_apply and wiki_lint behavior on a real vault; updating the PR body should trigger a fresh ClawSweeper review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +4, Tests +12. Total +16 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 13 9 +4
Tests 1 12 0 +12
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 25 9 +16

What I checked:

Likely related people:

  • vincentkoc: Git history shows Vincent Koc introduced and maintained the memory-wiki compile, lint, apply, and markdown stack, and current blame points to the same files. (role: feature owner and recent area contributor; confidence: high; commits: 516a43f9f28c, 9ce4abfe558e, 5716d83336fd; files: extensions/memory-wiki/src/markdown.ts, extensions/memory-wiki/src/compile.ts, extensions/memory-wiki/src/apply.ts)
  • steipete: Git history shows Peter Steinberger made recent memory-wiki and shared string-helper refactors touching adjacent parsing and normalization behavior. (role: adjacent area contributor; confidence: medium; commits: dffa88f39615, 59eb291c6e1c, 560a7aecd079; files: extensions/memory-wiki/src/markdown.ts, extensions/memory-wiki/src/compile.ts)
  • zhouhe-xydt: Git history shows a prior focused memory-wiki markdown test change around Unicode slug behavior, adjacent to this parser test surface but not the compile/apply root cause. (role: adjacent markdown test contributor; confidence: low; commits: 879bb5dd911e; files: extensions/memory-wiki/src/markdown.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jun 24, 2026
@hanZeng-08 hanZeng-08 closed this Jun 24, 2026
This was referenced Jun 24, 2026
Closed
Merged
@steipete

steipete commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

The canonical fix for #96125 has now landed via #97177 in commit 1289abddcb966cfa1535c7b82a08f555d9848445.

This candidate remains correctly closed: catching in the shared parser could erase malformed target metadata, while the landed version isolates vault scans and keeps every Markdown writer fail-closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

extensions: memory-wiki merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: wiki_apply & wiki_lint YAML 解析错误

2 participants

@hanZeng-08 @steipete