Skip to content

Comments

[ty] Support LSP go-to with vendored typeshed stubs#19057

Merged
ibraheemdev merged 1 commit intomainfrom
ibraheem/go-to-typeshed
Jul 2, 2025
Merged

[ty] Support LSP go-to with vendored typeshed stubs#19057
ibraheemdev merged 1 commit intomainfrom
ibraheem/go-to-typeshed

Conversation

@ibraheemdev
Copy link
Member

@ibraheemdev ibraheemdev commented Jun 30, 2025

Summary

Extracts the vendored typeshed stubs lazily and caches them on the local filesystem to support go-to in the LSP.

Resolves astral-sh/ty#77.

Test Plan

go-to-vendored.mp4

@ibraheemdev ibraheemdev force-pushed the ibraheem/go-to-typeshed branch from b2ece26 to b0689e9 Compare June 30, 2025 23:39
@ibraheemdev ibraheemdev changed the title Support LSP go-to with vendored typeshed stubs [ty] Support LSP go-to with vendored typeshed stubs Jun 30, 2025
@ibraheemdev ibraheemdev added ty Multi-file analysis & type inference server Related to the LSP server labels Jun 30, 2025
@ibraheemdev ibraheemdev force-pushed the ibraheem/go-to-typeshed branch from b0689e9 to dc1c401 Compare July 1, 2025 00:05
@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2025 •
edited
Loading

mypy_primer results

Changes were detected when running on open source projects 
pegen (https://github.com/we-like-parsers/pegen)
- TOTAL MEMORY USAGE: ~41MB
+ TOTAL MEMORY USAGE: ~45MB

beartype (https://github.com/beartype/beartype)
- TOTAL MEMORY USAGE: ~97MB
+ TOTAL MEMORY USAGE: ~88MB

dulwich (https://github.com/dulwich/dulwich)
- TOTAL MEMORY USAGE: ~142MB
+ TOTAL MEMORY USAGE: ~156MB

alerta (https://github.com/alerta/alerta)
- TOTAL MEMORY USAGE: ~106MB
+ TOTAL MEMORY USAGE: ~117MB

discord.py (https://github.com/Rapptz/discord.py)
-     memo fields = ~207MB
+     memo fields = ~189MB

jinja (https://github.com/pallets/jinja)
- TOTAL MEMORY USAGE: ~106MB
+ TOTAL MEMORY USAGE: ~97MB

tornado (https://github.com/tornadoweb/tornado)
-     memo fields = ~142MB
+     memo fields = ~129MB

pwndbg (https://github.com/pwndbg/pwndbg)
- TOTAL MEMORY USAGE: ~228MB
+ TOTAL MEMORY USAGE: ~207MB
-     memo fields = ~189MB
+     memo fields = ~171MB

pytest (https://github.com/pytest-dev/pytest)
-     memo fields = ~189MB
+     memo fields = ~207MB

openlibrary (https://github.com/internetarchive/openlibrary)
-     memo fields = ~189MB
+     memo fields = ~171MB

rotki (https://github.com/rotki/rotki)
-     memo fields = ~445MB
+     memo fields = ~490MB

@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2025 •
edited
Loading

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

@ibraheemdev ibraheemdev force-pushed the ibraheem/go-to-typeshed branch 2 times, most recently from ba5f5d9 to 81add77 Compare July 1, 2025 00:46
sharkdp
sharkdp approved these changes Jul 1, 2025
View reviewed changes
Copy link
Contributor

@sharkdp sharkdp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks (and works) great! Thank you.

dhruvmanila
dhruvmanila approved these changes Jul 1, 2025
View reviewed changes
Copy link
Member

@dhruvmanila dhruvmanila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great start!

The approach of using a file on disk on-demand is great especially given that it'll automatically provide the user with full LSP capabilities on those vendored files as well.

The one, very small, downside that I see is that the server will also store the file content in the Index while the file is open in the editor to keep track of the open files. This is done because whenever this file will be opened in the editor, the server will receive the didOpen notification from the client.

In the future, we might also want to make sure that these files are immutable by the system given that a user can apply any code actions if there are any.

Do we have any rough idea on how we want to solve this in the playground? Is it possible to use the same approach there?

@AlexWaygood AlexWaygood removed their request for review July 1, 2025 09:30
@dhruvmanila
Copy link
Member

dhruvmanila commented Jul 1, 2025

I just saw this in VS Code extension document: https://code.visualstudio.com/api/extension-guides/virtual-documents which, I think, is what Sorbet is doing as well from reading:

In the Sorbet VS Code extension: a TextDocumentContentProvider to present a Virtual Document.

This solution is limited to the clients that would implement this extension endpoint which for now would be just VS Code.

I don't think we should change the implementation as I believe the current solution is more robust in the sense that it doesn't require additional support on the client side.

@ibraheemdev ibraheemdev force-pushed the ibraheem/go-to-typeshed branch from 81add77 to 61fd717 Compare July 2, 2025 11:25
@ibraheemdev
Copy link
Member Author

ibraheemdev commented Jul 2, 2025 •
edited
Loading

Do we have any rough idea on how we want to solve this in the playground? Is it possible to use the same approach there?

I'm not too familiar with how the playground works, but I imagine we could create a in-memory "file" and display it in a new tab, yeah.

@ibraheemdev ibraheemdev merged commit ebc70a4 into main Jul 2, 2025
36 checks passed
@ibraheemdev ibraheemdev deleted the ibraheem/go-to-typeshed branch July 2, 2025 11:59
@ibraheemdev ibraheemdev mentioned this pull request Jul 2, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sharkdp sharkdp sharkdp approved these changes

@dhruvmanila dhruvmanila dhruvmanila approved these changes

@carljm carljm Awaiting requested review from carljm carljm is a code owner

@MichaReiser MichaReiser Awaiting requested review from MichaReiser MichaReiser is a code owner

@dcreager dcreager Awaiting requested review from dcreager dcreager is a code owner

Assignees

No one assigned

Labels

server Related to the LSP server ty Multi-file analysis & type inference

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Go to with vendored typeshed stubs

3 participants

@ibraheemdev @dhruvmanila @sharkdp