This causes a change in the snapshot test for Python syntax highlighting which I'm struggling to understand.

Before

After

It looks like this is highlighting 'range' as type.builtin rather than function.call.

But if you comment out some of the code above, now the highlighting is the same:

I think this query_syntax_tree method hasn't been working as intended for a while. It looks like the 'range' arguments were removed from Query.captures back in tree-sitter v0.23.0.

[EDIT: Looking closer, it doesn't look like Textual ever actually used the range arguments in this method. I'm not sure whether to just remove them or try to fix this? I decided worth fixing anyway in case it might help future improvements]

For reference here's the latest snapshot report from CI: https://github.com/Textualize/textual/actions/runs/16452663173/artifacts/3590409485

The failed snapshots look fine for the most part (and actually fixed a few bugs), but unfortunately there seems to be some quirks as mentioned above.

It looks like the syntax highlighting was possibly already broken...

PYTHON_EXAMPLE = """\
# Uncomment the line below
# foo = 69

print("Hello world")
x = range(10)
"""


from textual.app import App, ComposeResult
from textual.widgets import TextArea


class TextAreaSnapshot(App):
    def compose(self) -> ComposeResult:
        yield TextArea.code_editor(
            PYTHON_EXAMPLE,
            language="python",
        )


app = TextAreaSnapshot()
if __name__ == "__main__":
    app.run()

I think I'm finally starting to understand the issues with the highlighting.

Consider this Python code:

print("Hello world")
x = range(10)

The tree-sitter query captures (on this branch) look something like this.¹ Notice that function.call is before type.builtin, since 'print' was matched first.

[('variable', ['print', 'range', 'x']),
 ('function.call', ['print', 'range']),
 ('function.builtin', ['print', 'range']),
 ('punctuation.bracket', ['(', '(', ')', ')']),
 ('string', ['"Hello world"']),
 ('operator', ['=']),
 ('type.builtin', ['range']),
 ('number', ['10'])]

After removing the print function:

x = range(10)

Notice how the order of the captures has changed, where now function.call is after type.builtin

[('variable', ['x', 'range']),
 ('operator', ['=']),
 ('type.builtin', ['range']),
 ('function.call', ['range']),
 ('function.builtin', ['range']),
 ('punctuation.bracket', ['(', ')']),
 ('number', ['10'])]

The TextArea loops over the captures to build the 'highlight map':

When the TextArea lines are rendered, it loops over the line highlights to apply the syntax highlighting. This explains why 'range' will be highlighted differently depending on whether there's another function call before it!

I've updated the snapshots and marked this as ready for review.

Since it turns out the syntax highlighting is already broken, I think better to prioritise fixing the tree-sitter crash and revisit these issues later. Bumping tree-sitter actually fixes some issues with incorrect highlighting.

Note I still need to update the CHANGELOG if this is approved. This might be considered a breaking change since the syntax extras would now require Python >=3.10,

Thanks for putting in the leg work on this one. Will get this in the new release (out soon).

Thanks, Tom

No problem. Sorry I didn't manage to get back to the CHANGELOG before this was merged - is it worth adding something now (and maybe also the release notes)?