Skip to content

Commit e3a3c54

Browse files
mvanhornclaude
andauthored
Fix Haskell lexer: handle escape sequences in character literals (#3069)
* fix: handle escape sequences in Haskell character literals The root-state pattern for character literals only matched single non-backslash characters like 'a'. Escape sequences like '\n', '\t', '\\' were incorrectly tokenized as Keyword.Type + Name fragments. Added a pattern for '\.' (backslash + any char) to match escape character literals, placed after the existing non-escape pattern. Fixes #1795 Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add snippet test for escape character literals Covers '\n', '\t', '\\', and 'a' tokenization as Literal.String.Char. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: regenerate Haskell and LHS golden outputs Update expected token outputs for example.hs and Sudoku.lhs to reflect the new escape character literal tokenization. --------- Co-authored-by: Matt Van Horn <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]>
1 parent d7c3453 commit e3a3c54

4 files changed

Lines changed: 22 additions & 6 deletions

File tree

‎pygments/lexers/haskell.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,8 @@ class HaskellLexer(RegexLexer):
5353
(r'\bmodule\b', Keyword.Reserved, 'module'),
5454
(r'\berror\b', Name.Exception),
5555
(r'\b({})(?!\')\b'.format('|'.join(reserved)), Keyword.Reserved),
56-
(r"'[^\\]'", String.Char), # this has to come before the TH quote
56+
(r"'[^\\]'", String.Char), # character literal
57+
(r"'\\.'", String.Char), # escape character literal (e.g. '\n')
5758
(r'^[_' + uni.Ll + r'][\w\']*', Name.Function),
5859
(r"'?[_" + uni.Ll + r"][\w']*", Name),
5960
(r"('')?[" + uni.Lu + r"][\w\']*", Keyword.Type),

‎tests/examplefiles/haskell/example.hs.output‎

Lines changed: 1 addition & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎tests/examplefiles/lhs/Sudoku.lhs.output‎

Lines changed: 1 addition & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---input---
2+
'\n'
3+
'\t'
4+
'\\'
5+
'a'
6+
7+
---tokens---
8+
"'\\n'" Literal.String.Char
9+
'\n' Text.Whitespace
10+
11+
"'\\t'" Literal.String.Char
12+
'\n' Text.Whitespace
13+
14+
"'\\\\'" Literal.String.Char
15+
'\n' Text.Whitespace
16+
17+
"'a'" Literal.String.Char
18+
'\n' Text.Whitespace

0 commit comments

Comments
 (0)