You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
markdown: predicted heading slugs diverge from GitHub's actual anchors (emoji VS16 dropped, No-category characters kept), so internal links to those headings break #190
resolve_anchors binds a link to a heading-coincident anchor to the fragment the heading "will have in the rendered Markdown" (src/render/markdown/anchors.rs:1-5). For that binding to work, gfm_slug has to reproduce the anchor GitHub actually generates. It doesn't for two classes of characters, in opposite directions, and every internal link to such a heading lands on a fragment GitHub never assigned.
gfm_slug (src/render/markdown/anchors.rs:180) keeps only is_alphanumeric, the hardcoded mark ranges in is_combining_mark (anchors.rs:199), connector punctuation, and hyphens. U+FE0F (VARIATION SELECTOR-16) is in none of those, so it is filtered out. GitHub's anchor algorithm (github-slugger, which GitHub renders with) removes a fixed punctuation set; VS16 is not in it, so the anchor keeps the invisible selector.
² (U+00B2) and ½ (U+00BD) pass Rust's char::is_alphanumeric (category No is numeric), so gfm_slug keeps them. github-slugger removes both, so the GitHub anchor does not contain them.
Repro
Minimal .docx (styles.xml maps Heading1, bookmark inside each heading, w:hyperlink w:anchor=... after it), converted with 0.2.4 (commit 261fc25, identical to v0.2.4):
# ❤️ Love → [go to heading](#-love) GitHub anchor: #️-love (fe0f kept)
# 2️⃣ Second → [go to second](#2⃣-second) GitHub anchor: #2️⃣-second (fe0f kept)
# x² formula → [go to x](#x²-formula) GitHub anchor: #x-formula (00b2 dropped)
Exact fragments from the conversion (hex-escaped):
vs github-slugger 2.0.0, which reproduces GitHub's anchors. Heading text such as ## ⚠️ Deprecation notice or ## ✨ What's new is common in real documents, so the wrong fragment is emitted for exactly the headings people decorate.
Verified against the npm build @firecrawl/anydoc 0.2.4 (source on main is unchanged: only README commits since v0.2.4).
Direction
gfm_slug's keep-list is close but not equal to github-slugger's remove-list. Matching github-slugger exactly for these two classes — keep U+FE0F (it is a mark GitHub's slugger preserves), and drop the No/Nl characters github-slugger removes (or, more robustly, derive the keep predicate from the same character set github-slugger removes) — would fix both directions. A differential test over a corpus of tricky headings against github-slugger would keep this property from regressing.
Activity
fix(markdown): match GitHub's anchor removal set exactly (firecrawl#190)