fix(wiki): drop a leading BOM on a page with no frontmatter - #663

What changed

markdown::parse strips a leading UTF-8 BOM before it looks for the frontmatter fence, but the no-frontmatter path returned the untouched input as the body instead of the stripped trimmed (crates/ai-memory-wiki/src/markdown.rs:34-52). A page with a BOM and no frontmatter therefore kept the mark in front of its first line, where it is not a byte-order mark any more. It now returns trimmed on both paths, which is what the frontmatter path already did.

Before: parse("\u{FEFF}# Hand written\n\nBody.\n").body == "\u{FEFF}# Hand written\n\nBody.\n". After: "# Hand written\n\nBody.\n". A page that already had frontmatter is unchanged, and so is every page with no BOM.

Why

Two observable consequences.

derive_title picks a title by scanning body lines for a # prefix. With the mark sitting in front of the # the H1 stops matching, so reindex_page and restore_page_from_checkpoint index the page under its filename stem rather than its heading. A hand-edited notes/hand-written.md starting # Hand written came back as "hand-written".

The OKF v0.2 file pass is worse, because it writes. Its contract is "conform every wiki .md in place (body untouched)" (crates/ai-memory-wiki/src/migrations/m2026_09_okf_conformance.rs:8), and conform_file implements that by parsing, filling the frontmatter, and writing emit(&md) back through an atomic rename. Fed a BOM-prefixed page with no frontmatter it produced:

---
type: Note
generated:
  by: process:ai-memory/2.1.0
  at: 2026-09-06T22:38:30Z
---
\u{feff}# Hand written

Body.

The mark is now after the closing fence, inside the body, where it is a stray zero-width no-break space ahead of the H1 rather than a byte-order mark. That is a permanent edit to the source of truth, and the migration is one-shot.

The wiki is meant to be hand-editable and the project supports Windows, where a BOM is what several editors and PowerShell write by default. The CLI hook path already handles this deliberately (crates/ai-memory-cli/src/commands/hook.rs:822, docs/windows.md:512); this is the same class of input reaching the wiki parser.

Test plan

• cargo fmt --all -- --check passes
• git diff --check passes
• cargo clippy --workspace --all-targets -- -D warnings passes
• cargo test --workspace --all-targets passes
• Manual test: the workspace run is 3108 passed / 0 failed across 15 test binaries, ai-memory-wiki 158 of them. With the two new tests kept and the one-word fix reverted, parses_bom_prefixed_body_without_frontmatter fails with left: "\u{feff}# Hand written\n\nBody.\n" and conforming_a_bom_prefixed_page_does_not_move_the_mark_into_the_body fails printing the conformed file with the mark after the fence. The pre-existing parses_bom_prefixed_frontmatter passes either way, so the frontmatter path is untouched.

Commit attribution

• I verified the name and email on every commit in this PR and corrected any unintended identity before requesting merge. See commit attribution guidance.

Release impact

• Patch: bug fix, no new surface
• Minor: additive: new flag/subcommand/MCP tool/config key, new agent harness or provider
• Major (breaking): on-disk format, removed/renamed surface, breaking MCP schema change

CHANGELOG (merge gate)

• I added a CHANGELOG.md [Unreleased] entry under ### Fixed.
• Any changed default is called out in "What changed" above. Nothing configurable changed; the only behaviour difference is on input that carries a leading BOM.

Notes for reviewers

The tests live in the two modules that own the two symptoms: the parse-boundary one next to the existing parses_bom_prefixed_frontmatter in markdown.rs, and the file-pass one in the migration's own test module, calling conform_file directly rather than standing up a store.

One knock-on worth naming: a store that already indexed a BOM-prefixed frontmatter-less page will supersede it once on the next reindex, because the body sha256 changes when the mark leaves. That is the fix landing, and it corrects the title on the same pass.

I did not touch the ---\r\n case. A CRLF file is not recognised as having frontmatter at all today, which is a separate decision about what "the canonical ---\n<yaml>\n---\n block" means, and it would change what parses rather than what a parsed body contains.