Skip to content

Commit 52e9844

Browse files
committed
feat(md088): normalize quotes and dashes to ASCII (#763)
2 parents 5db8f0c + d027161 commit 52e9844

19 files changed

Lines changed: 784 additions & 184 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ Python linting, rumdl brings similar speed and developer experience improvements
5252
It offers:
5353

5454
- ⚡️ **Built for speed** with Rust - significantly faster than alternatives
55-
- 🔍 **<!-- RULE_COUNT -->81<!-- /RULE_COUNT --> lint rules** covering common Markdown issues
55+
- 🔍 **<!-- RULE_COUNT -->82<!-- /RULE_COUNT --> lint rules** covering common Markdown issues
5656
- 🛠️ **Automatic formatting** with `--fix` for files and stdin/stdout
5757
- 📦 **Zero dependencies** - single binary with no runtime requirements
5858
- 🔧 **Highly configurable** with TOML-based config files
@@ -526,7 +526,7 @@ The action ref (`rvben/rumdl@v0`) selects the GitHub Action version, while the o
526526

527527
## Rules
528528

529-
rumdl implements <!-- RULE_COUNT -->81<!-- /RULE_COUNT --> lint rules for Markdown files. Here are some key rule categories:
529+
rumdl implements <!-- RULE_COUNT -->82<!-- /RULE_COUNT --> lint rules for Markdown files. Here are some key rule categories:
530530

531531
| Category | Description | Example Rules |
532532
| -------------- | ---------------------------------------- | ------------------- |

‎docs/comparison.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ For detailed comparisons with specific tools, see:
1313

1414
| Tool | Type | Language | Rules | Auto-fix | Flavors | Config format | Plugins | LSP |
1515
| --------------------- | ------------- | -------- | ----------------------------------------- | -------- | ------- | ----------------------- | ------------ | --- |
16-
| **rumdl** | Lint + Format | Rust | <!-- RULE_COUNT -->81<!-- /RULE_COUNT --> | Yes | 9 | TOML, JSON, YAML | No | Yes |
16+
| **rumdl** | Lint + Format | Rust | <!-- RULE_COUNT -->82<!-- /RULE_COUNT --> | Yes | 9 | TOML, JSON, YAML | No | Yes |
1717
| **markdownlint-cli** | Lint | Node.js | 53 | Yes | No | JSON, JSONC, YAML, TOML | Yes (JS) | No |
1818
| **markdownlint-cli2** | Lint | Node.js | 53 | Yes | No | JSONC, YAML, JS | Yes (JS) | No |
1919
| **remark-lint** | Lint | Node.js | ~80 (via presets) | No | No | JS, JSON, YAML | Yes (JS) | No |
@@ -33,7 +33,7 @@ cover common cases. No built-in auto-fix for lint violations (remark itself can
3333

3434
**mado** is a Rust-based linter with 38 rules (33 stable, 5 unstable). It has no auto-fix and no plugin system.
3535

36-
**rumdl** implements all 53 markdownlint rules plus <!-- RULE_COUNT_ADDITIONAL -->28<!-- /RULE_COUNT_ADDITIONAL --> additional rules (<!-- RULE_COUNT -->81<!-- /RULE_COUNT --> total). It supports
36+
**rumdl** implements all 53 markdownlint rules plus <!-- RULE_COUNT_ADDITIONAL -->29<!-- /RULE_COUNT_ADDITIONAL --> additional rules (<!-- RULE_COUNT -->82<!-- /RULE_COUNT --> total). It supports
3737
auto-fix for most rules and includes rules not found in other tools, such as relative link validation (MD057), footnote checks (MD066-MD068), nested code fence detection (MD070), and TOC validation
3838
(MD073).
3939

@@ -136,7 +136,7 @@ Cold start benchmark on the Rust Book (478 Markdown files), measured with [hyper
136136
137137
**Notes:**
138138

139-
- **mado** is faster in cold-start benchmarks because it does less work per file: fewer rules (38 vs <!-- RULE_COUNT -->81<!-- /RULE_COUNT -->), no fix generation, and no flavor detection.
139+
- **mado** is faster in cold-start benchmarks because it does less work per file: fewer rules (38 vs <!-- RULE_COUNT -->82<!-- /RULE_COUNT -->), no fix generation, and no flavor detection.
140140
The gap reflects feature surface area, not implementation quality.
141141
- **rumdl** supports result caching (`rumdl check` without `--no-cache`), which skips unchanged files on subsequent runs — typically under 50 ms, faster than mado's cold start.
142142
- **pymarkdown** performs well for a Python tool due to its efficient scanner architecture.
@@ -147,5 +147,5 @@ Cold start benchmark on the Rust Book (478 Markdown files), measured with [hyper
147147

148148
- [Comparison with markdownlint](markdownlint-comparison.md) — detailed rule-by-rule comparison and migration guide
149149
- [Comparison with mdformat](mdformat-comparison.md) — formatting feature comparison and migration guide
150-
- [Rules Reference](rules.md) — complete list of rumdl's <!-- RULE_COUNT -->81<!-- /RULE_COUNT --> rules
150+
- [Rules Reference](rules.md) — complete list of rumdl's <!-- RULE_COUNT -->82<!-- /RULE_COUNT --> rules
151151
- [Markdown Flavors](flavors.md) — flavor configuration and per-rule adjustments

‎docs/getting-started/quickstart.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,5 +145,5 @@ Each line shows:
145145
## Next Steps
146146

147147
- [CLI Commands](../usage/cli.md) - Full command reference
148-
- [Rules Reference](../rules.md) - Explore all <!-- RULE_COUNT -->81<!-- /RULE_COUNT --> rules
148+
- [Rules Reference](../rules.md) - Explore all <!-- RULE_COUNT -->82<!-- /RULE_COUNT --> rules
149149
- [Configuration](../global-settings.md) - Advanced configuration options

‎docs/index.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ icon: lucide/file-check
1616

1717
[:octicons-arrow-right-24: Benchmarks](#performance)
1818

19-
- :mag:{ .lg .middle } **<!-- RULE_COUNT -->81<!-- /RULE_COUNT --> lint rules**
19+
- :mag:{ .lg .middle } **<!-- RULE_COUNT -->82<!-- /RULE_COUNT --> lint rules**
2020

2121
---
2222

@@ -76,7 +76,7 @@ With intelligent caching, subsequent runs are even faster - rumdl only re-lints
7676
## Features
7777

7878
- :zap: **Built for speed** with Rust - significantly faster than alternatives
79-
- :mag: **<!-- RULE_COUNT -->81<!-- /RULE_COUNT --> lint rules** covering common Markdown issues
79+
- :mag: **<!-- RULE_COUNT -->82<!-- /RULE_COUNT --> lint rules** covering common Markdown issues
8080
- :wrench: **Automatic formatting** with `--fix` for files and stdin/stdout
8181
- :package: **Zero dependencies** - single binary with no runtime requirements
8282
- :gear: **Highly configurable** with TOML-based config files
@@ -100,7 +100,7 @@ With intelligent caching, subsequent runs are even faster - rumdl only re-lints
100100

101101
- [:octicons-book-24: **Rules Reference**](rules.md)
102102

103-
Explore all <!-- RULE_COUNT -->81<!-- /RULE_COUNT --> linting rules with examples.
103+
Explore all <!-- RULE_COUNT -->82<!-- /RULE_COUNT --> linting rules with examples.
104104

105105
- [:octicons-gear-24: **Configuration**](global-settings.md)
106106

‎docs/markdownlint-comparison.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,22 +10,22 @@ rumdl offers **high markdownlint compatibility with intentional differences** wh
1010

1111
- **Performance**: rumdl is significantly faster (30-100x in many cases) thanks to Rust and intelligent caching
1212
- **Rule Coverage**: All 53 markdownlint rules are implemented, with a small number of intentional behavioral differences documented below
13-
- **Unique Features**: <!-- RULE_COUNT_ADDITIONAL -->28<!-- /RULE_COUNT_ADDITIONAL --> additional rules (MD057, MD061-<!-- RULE_MAX -->MD087<!-- /RULE_MAX -->), built-in LSP server, VS Code extension, 6 Markdown flavors
13+
- **Unique Features**: <!-- RULE_COUNT_ADDITIONAL -->29<!-- /RULE_COUNT_ADDITIONAL --> additional rules (MD057, MD061-<!-- RULE_MAX -->MD088<!-- /RULE_MAX -->), built-in LSP server, VS Code extension, 6 Markdown flavors
1414
- **Configuration**: Automatic markdownlint config discovery and conversion
1515

1616
## Rule Coverage
1717

1818
### Implemented Rules
1919

20-
rumdl implements **<!-- RULE_COUNT -->81<!-- /RULE_COUNT --> rules total**: all 53 markdownlint rules plus <!-- RULE_COUNT_ADDITIONAL -->28<!-- /RULE_COUNT_ADDITIONAL --> unique rules.
20+
rumdl implements **<!-- RULE_COUNT -->82<!-- /RULE_COUNT --> rules total**: all 53 markdownlint rules plus <!-- RULE_COUNT_ADDITIONAL -->29<!-- /RULE_COUNT_ADDITIONAL --> unique rules.
2121

2222
**Markdownlint-compatible rules (53):** All markdownlint rules are implemented with full compatibility. See the [Rules Reference](rules.md) for the complete list.
2323

2424
**Note:** Rule numbers MD001-MD060 have gaps (MD002, MD006, MD008, MD015-MD017 were never implemented in markdownlint). rumdl maintains these gaps for compatibility.
2525

2626
### Rules Unique to rumdl
2727

28-
rumdl implements <!-- RULE_COUNT_ADDITIONAL -->28<!-- /RULE_COUNT_ADDITIONAL --> additional rules not found in markdownlint:
28+
rumdl implements <!-- RULE_COUNT_ADDITIONAL -->29<!-- /RULE_COUNT_ADDITIONAL --> additional rules not found in markdownlint:
2929

3030
| Rule | Name | Description |
3131
| ------ | ------------------------------ | ---------------------------------------------------------- |

‎docs/md088.md‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# MD086 - Normalize to ASCII quotes and dashes
2+
3+
Aliases: `normalize-quotes-dashes`
4+
5+
**Disabled by default.** This rule is opt-in: enable it explicitly with
6+
`extend-enable`.
7+
8+
## What this rule does
9+
10+
This rule detects Unicode quotes and dashes that look like their ASCII equivalents, and replaces them with plain ASCII characters.
11+
12+
By default, it will only replace multiple styles of single and double quotes by their simple ASCII equivalents.
13+
14+
You can optionally enable replacement for dashes.
15+
16+
Characters covered by this rule:
17+
18+
| Unicode Character Name | Character | Codepoint | ASCII Replacement |
19+
| ------------------------------------------ | --------- | --------- | ----------------- |
20+
| Modifier Letter Apostrophe | ʼ | U+02BC | ' |
21+
| Left Single Quotation Mark | ‘ | U+2018 | ' |
22+
| Right Single Quotation Mark | ’ | U+2019 | ' |
23+
| Single Low-9 Quotation Mark | ‚ | U+201A | ' |
24+
| Single High-Reversed-9 Quotation Mark | ‛ | U+201B | ' |
25+
| Prime | ′ | U+2032 | ' |
26+
| Left Double Quotation Mark | “ | U+201C | " |
27+
| Right Double Quotation Mark | ” | U+201D | " |
28+
| Double Low-9 Quotation Mark | „ | U+201E | " |
29+
| Double High-Reversed-9 Quotation Mark | ‟ | U+201F | " |
30+
| Double Prime | ″ | U+2032 | " |
31+
| Hyphen | ‐ | U+2010 | - |
32+
| Non-Breaking Hyphen | ‑ | U+2011 | - |
33+
| Figure Dash | ‒ | U+2012 | - |
34+
| En Dash | – | U+2013 | - |
35+
| Em Dash | — | U+2014 | - |
36+
| Horizontal Bar | ― | U+2015 | - |
37+
38+
This rule checks only prose content and skips:
39+
40+
- fenced code blocks
41+
- indented code blocks
42+
- inline code spans
43+
44+
## Why this matters
45+
46+
While more typographically correct, those characters only offer a very marginal readability gain.
47+
48+
Fancy quotes and dashes often come from smart punctuation features or
49+
copy/paste from rich text sources. These characters can create variability, noisy diffs and
50+
unexpected behavior in tooling that expects ASCII punctuation.
51+
52+
Bad handling of Unicode characters in editors/IDEs is also a source of "Mojibake",
53+
which are sequences of strange characters generated by decoding a Unicode-encoded
54+
character with another charset like ISO-8859-x or CP125x. See MD083 for a rule that
55+
detects such Mojibake.
56+
57+
Normalizing quotes and dashes to ASCII keeps Markdown content predictable and avoids
58+
hidden character drift across contributors and platforms.
59+
60+
## Examples
61+
62+
### Incorrect
63+
64+
```text
65+
It’s “quoted” text with ‛variantsʼ.
66+
```
67+
68+
### Correct
69+
70+
```text
71+
It's "quoted" text with 'variants'.
72+
```
73+
74+
### Auto-fix behavior in code
75+
76+
```text
77+
Prose ’quote’ and `code ’quote’`
78+
```
79+
80+
Fixes to:
81+
82+
```text
83+
Prose 'quote' and `code ’quote’`
84+
```
85+
86+
The inline code span is intentionally unchanged.
87+
88+
## Configuration
89+
90+
```toml
91+
[MD086]
92+
normalize-quotes = true
93+
normalize-dashes = false
94+
allow = []
95+
```
96+
97+
- `normalize-quotes`: if enabled (by default), replace Unicode quotes by simple ASCII ones.
98+
- `normalize-dashes`: if enabled, replace Unicode dashes by simple ASCII ones.
99+
- `allow`: A list of codepoints to preserve, written as the direct Unicode character,
100+
or as `U+XXXX`, `U+XXXXX`, or `U+XXXXXX` (for example `U+2019`).
101+
Any allowed codepoint is ignored by MD086 and will not be replaced.
102+
103+
Example:
104+
105+
```toml
106+
[MD086]
107+
enabled = true
108+
normalize-quotes = true
109+
normalize-dashes = true
110+
allow = ["U+2032", "U+2032"]
111+
```
112+
113+
With this configuration, the rule is enabled, quotes and dashes are replaced,
114+
excepted "prime" `′` and "double prime" `″`.

‎docs/mdformat-comparison.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ This document compares rumdl and mdformat, focusing on their formatting capabili
77
Both tools format Markdown files, but serve different purposes:
88

99
- **mdformat**: Pure formatter focused on consistent Markdown output
10-
- **rumdl**: Combined linter and formatter with <!-- RULE_COUNT -->81<!-- /RULE_COUNT --> rules plus formatting
10+
- **rumdl**: Combined linter and formatter with <!-- RULE_COUNT -->82<!-- /RULE_COUNT --> rules plus formatting
1111

1212
**Key Differences:**
1313

@@ -16,7 +16,7 @@ Both tools format Markdown files, but serve different purposes:
1616
| Primary purpose | Formatting only | Linting + formatting |
1717
| Language | Python | Rust |
1818
| Performance | Good | Faster (native + caching) |
19-
| Linting rules | ❌ | ✅ <!-- RULE_COUNT -->81<!-- /RULE_COUNT --> rules |
19+
| Linting rules | ❌ | ✅ <!-- RULE_COUNT -->82<!-- /RULE_COUNT --> rules |
2020
| Extensibility | Plugin ecosystem | Built-in flavors |
2121
| CommonMark | Strict compliance | Strict compliance |
2222

@@ -47,7 +47,7 @@ Both tools format Markdown to a consistent style:
4747

4848
### Linting (rumdl only)
4949

50-
rumdl provides <!-- RULE_COUNT -->81<!-- /RULE_COUNT --> linting rules that mdformat does not have:
50+
rumdl provides <!-- RULE_COUNT -->82<!-- /RULE_COUNT --> linting rules that mdformat does not have:
5151

5252
- **Broken link detection** (MD051, MD052, MD057)
5353
- **Accessibility checks** (MD045 - image alt text)
@@ -299,7 +299,7 @@ style = "atx"
299299
| Capability | mdformat | rumdl |
300300
| ----------------------- | ------------------ | ---------------------- |
301301
| Markdown formatting | ✅ Primary focus | ✅ Via `rumdl fmt` |
302-
| Markdown linting | ❌ | ✅ <!-- RULE_COUNT -->81<!-- /RULE_COUNT --> rules |
302+
| Markdown linting | ❌ | ✅ <!-- RULE_COUNT -->82<!-- /RULE_COUNT --> rules |
303303
| Performance | Good | Faster (native binary) |
304304
| Extended syntax | Plugins | Built-in flavors |
305305
| Editor integration | Basic | LSP + VS Code |

‎docs/rules.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
## Introduction
66

7-
rumdl implements <!-- RULE_COUNT -->81<!-- /RULE_COUNT --> rules for checking Markdown files. This document provides a comprehensive reference of all available rules, organized by category.
7+
rumdl implements <!-- RULE_COUNT -->82<!-- /RULE_COUNT --> rules for checking Markdown files. This document provides a comprehensive reference of all available rules, organized by category.
88
Each rule has a brief description and a link to its detailed documentation.
99

1010
For information on global configuration settings (file selection, rule enablement, etc.), see the [Global Settings Reference](global-settings.md).
@@ -284,6 +284,7 @@ Severity affects:
284284
| [MD083](md083.md) | Mojibake | Detect mojibake due to encoding issues |
285285
| [MD086](md086.md) | No unclosed comments | Comment openers should have a closer |
286286
| [MD087](md087.md) | Unused disable comment | Disable comments should suppress something |
287+
| [MD088](md088.md) | Quotes and dashes | Normalize Quotes and Dashes |
287288

288289
## Using Rules
289290

‎src/config/registry.rs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -279,6 +279,7 @@ pub static RULE_ALIAS_MAP: phf::Map<&'static str, &'static str> = phf::phf_map!
279279
"MD085" => "MD085",
280280
"MD086" => "MD086",
281281
"MD087" => "MD087",
282+
"MD088" => "MD088",
282283

283284
// Aliases (hyphen format)
284285
"HEADING-INCREMENT" => "MD001",
@@ -366,6 +367,7 @@ pub static RULE_ALIAS_MAP: phf::Map<&'static str, &'static str> = phf::phf_map!
366367
"PARAGRAPH-CONTINUATION-INDENT" => "MD085",
367368
"NO-UNCLOSED-COMMENTS" => "MD086",
368369
"UNUSED-DISABLE-COMMENT" => "MD087",
370+
"QUOTES-DASHES" => "MD088",
369371
};
370372

371373
/// The name rumdl uses when it writes a rule name itself, one per rule.
@@ -456,6 +458,7 @@ pub static RULE_PRIMARY_ALIAS: phf::Map<&'static str, &'static str> = phf::phf_m
456458
"MD085" => "paragraph-continuation-indent",
457459
"MD086" => "no-unclosed-comments",
458460
"MD087" => "unused-disable-comment",
461+
"MD088" => "quotes-dashes",
459462
};
460463

461464
/// The readable name for a rule ID, or `None` for a name that is not a rule ID.

‎src/lint_context/mod.rs‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1132,6 +1132,14 @@ impl<'a> LintContext<'a> {
11321132
self.is_in_reference_def(pos)
11331133
}
11341134

1135+
/// Check if `pos`` is within a bare URL
1136+
pub fn is_in_bare_url(&self, pos: usize) -> bool {
1137+
let bare_urls = self.bare_urls();
1138+
// Binary search (sorted by byte_offset) for the candidate containing byte_pos
1139+
let idx = bare_urls.partition_point(|url| url.byte_offset <= pos);
1140+
idx > 0 && pos < bare_urls[idx - 1].byte_end
1141+
}
1142+
11351143
/// Get parsed inline configuration state.
11361144
pub fn inline_config(&self) -> &InlineConfig {
11371145
&self.inline_config

0 commit comments

Comments
 (0)