Skip to content

Semantic tokens for doc comments code #63739

Closed

Description

@FMorschel

Before the server starts and sends semantic tokens for VS Code, this is what I see for a file on a public project I have:

Image

After the server started and sent semantic tokens, this is what I see:

Image

Note that line 21 (single word between quotes) and lines 28 to 34 are originally highlighted differently.

Can we make that distinction with semantic tokens too?

CC @DanTup

Activity

  1. added
    area-devexpFor issues related to the analysis server, IDE support, linter, `dart fix`, and diagnostic messages.
    on Jul 2, 2026
  2. DanTup commented on Jul 2, 2026

    @DanTup
    Collaborator

    I think this is similar to Dart-Code/Dart-Code#3330 and blocked on dart-lang/tools#1287.

    The textmate grammar does its own parsing (with regex) and attempts to detect codeblocks (I don't think it's 100% reliable, although its results are also replaced with semantic tokens a lot of the time so issues might not be obvious).

    Server (AFAIK) doesn't currently parse the markdown, and the markdown package doesn't provide offsets/lengths to know where each kind of node is.

    That said, if the package:markdown work is non-trivial, triple backtick start/ends might be easier to detect than other blocks in the comment, so maybe we could do some basic parsing in the server for that (cc @bwilkerson for his opinion).

  3. FMorschel commented on Jul 2, 2026

    @FMorschel
    ContributorAuthor

    This parsing (or at least part of it) is already done, see this for a case concerning a code block:

  4. DanTup commented on Jul 2, 2026

    @DanTup
    Collaborator

    Oh, you're right - it appears that Comment already contains codeBlocks alongside the references, so maybe everything we need is already there (for code blocks as described here at least, just not the backtick-quoted identifiers in the other issue linked above). I'll see if it's an easy fix.

  5. self-assigned this
    on Jul 2, 2026
  6. DanTup commented on Jul 2, 2026

    @DanTup
    Collaborator
  7. FMorschel commented on Jul 2, 2026

    @FMorschel
    ContributorAuthor

    @pq just in case the Dart IJ plugin needs to update anything to make this work.

  8. DanTup commented on Jul 2, 2026

    @DanTup
    Collaborator

    The changes I made were specifically only to LSP Semantic Tokens, since we'd need to add a new HighlightRegionType to support this for the legacy protocol, and that would then also need changes in IntelliJ (I'm assuming IntelliJ uses syntax highlighting from the server, but I'm not certain). Given the goal to move to IntelliJ to LSP, I doubt that's worthwhile.

    That said, I'm not sure whether IntelliJ also has some client-side syntax highlighting, equivalent to the textmate grammar in VS Code, and it might make sense to map some of the servers custom modifiers so they are coloured (for example there is a Dart-Code change at Dart-Code/Dart-Code@9fb785b that goes with this server change, mapping the new custom modifier onto the same textmate scope that the textmate grammar uses for code blocks).

    (@pq @helin24 if anything here isn't clear, I can provide more details when it's appropriate)

  9. pq commented on Jul 6, 2026

    @pq
    Member

    Taking a quick look at the Dart grammar in IntelliJ, I don't see comments being parsed so I think this is all server (see e.g., : HighlightRegion.java and HighlightRegionType.java). @helin24: maybe this is something to keep on the radar as we tackle migrating semantic highlighting to LSP?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-devexpFor issues related to the analysis server, IDE support, linter, `dart fix`, and diagnostic messages.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions