-
Notifications
You must be signed in to change notification settings - Fork 766
Improve Typst Lexer #2724
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve Typst Lexer #2724
Conversation
|
Thanks @Nerixyz for your continued efforts on this topic! |
oliverpool
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the work, it looks good!
(note: not a maintainer, just a random user)
Oopsie, I should've read the docs better. Thank you for pointing that out! |
|
That was quick 😄 I imported your lexer into chroma and tested it with the following snippet: #let song(
title: none,
title_index: none,
singer: none,
singer_index: none,
references: (),
line_color: rgb(0xd0, 0xd0, 0xd0),
header_display: (number, title, singer) => grid(
columns: (auto,1fr),
inset: 4pt,
align: horizon,
grid.cell(fill: rgb(0xd0, 0xd0, 0xd0),align: top, inset:(rest:4pt, /*right: 10pt*/), heading(outlined: false,str(number))),
[
#heading(title)
#if singer != none {
strong(singer) + "\n"
}
],
),
// comment
doc,
)the This may be due to the conversion to chroma's xml (I don't know how to use pygment to quickly render a snippet, so maybe this issue is not present here), but maybe you can reproduce it/test it as well? |
e39e73d to
1d48b45
Compare
I missed that. Should be supported now (although with some quirks): #if foo {
bar
} else {
baz
} qoxIn this case, |
1d48b45 to
d3549cb
Compare
Co-Authored-By: oliverpool <[email protected]>
d3549cb to
2fcb426
Compare
oliverpool
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the latest changes, my own typst examples look very good 👍 (I don't use math, but a good mix of markup, content and functions).
Thank you for providing feedback! I also use it on https://nerixyz.github.io/icu-typ/ (mostly code). If there are any further issues, feel free to @ me. |
|
I'll trust you guys you know what you're doing here, the lexing looks fine. |

This PR advances the Typst lexer. It's extended from #2656 by @jonaspleyer (see #2656 (comment)).
It adds a few snippets and most importantly, the example is now tokenized correctly. The example was fixed too (#2596 (comment)).
I described all commits I made in #2656 (comment) (which mainly worked its way towards correctly handling the example).
It might be nice to add an option for the lexer to start incode(similar to thestartinlineoption for PHP) - I'm not sure what the best implementation of that would be, though.I've added the
startoption, which specifies the starting state of the lexer (one of 'markup', 'code', 'math') - this can be useful when highlighting Typst code inline.Original description of #2656: