There are many aspects of the semantic token feature where there is no objectively "correct" answer. For example, if a symbol is a variable whose type is declared as Callable, should it be a "variable" or "function" token? The "correct" answer to these questions typically comes down to "whatever is preferred by the most users". That means this feature will require adjustments over time based on user feedback. In that sense, this feature will never be "complete".

The functionality in the current implementation relies on some semantic analysis features that are not yet complete. The most notable one is support for self and cls parameters and attributes accessed from self or cls. Once the semantic analyzer is able to infer types of expressions like self.foo, the semantic token implementation will start classifying these attributes correctly.

Here's a checklist of additional work that will likely need to be done.

Improve classification for name tokens that are imported from other modules. Currently, these are classified based on their types, which often means they're classified as variables when they should be classes, etc.
Properly handle Annotated type annotations. All type arguments to an Annotated type form other than the first should be treated as value expressions, not as type expressions. Currently, the code blindly treats all type arguments within all annotations as type expressions.
Properly categorize names that are parameters as either parameter ✅ , selfParameter ✅ or clsParameter tokens.
Properties (or perhaps more generally, descriptor objects?) defined within a class should be classified as a property token rather than just a variable.
Special forms like Protocol and TypedDict should probably be classified as class tokens (rather than variable tokens), but this is debatable. Could wait for user feedback.
Type aliases (including those defined within the Python 3.12 type statement) do not currently have a dedicated semantic token type, but maybe they should? Could wait for user feedback.
Additional token modifiers could be added (e.g. for static methods, abstract methods, abstract classes, etc.)? Could wait for user feedback.

Semantic tokens #791

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions