[ty] Add "Did you mean" suggestions for stdlib import typos#23341
[ty] Add "Did you mean" suggestions for stdlib import typos#23341AlexWaygood wants to merge 4 commits intomainfrom
Conversation
…4p0CV # Conflicts: # crates/ty_python_semantic/src/types/infer/builder.rs
Typing conformance resultsNo changes detected ✅ |
|
Memory usage reportMemory usage unchanged ✅ |
5815267 to
913bb0e
Compare
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
unresolved-import |
0 | 0 | 1,314 |
invalid-assignment |
0 | 0 | 1 |
| Total | 0 | 0 | 1,315 |
913bb0e to
93caf6e
Compare
sharkdp
left a comment
There was a problem hiding this comment.
we can give suggestions here that are generally pretty good
I realize that this is not a fair sample, but in all of the 1300 ecosystem cases, all that this does is to add noise to the diagnostic message. Those are all cases where a dependency is not properly installed/available and the "Did you mean …" suggesting is only distracting from the actual solution.
Obviously, if you mistype a package name, this diagnostic can be helpful. But arguably, developers do this less and less. Even if they work without AI, they probably use autocompletion and/or auto-imports, where those "Did you mean …" suggestions can only be wrong/misleading.
So while this is certainly helpful for the subset of developers that still type out import pydatnic by hand (sic), I'm not completely convinced this is a net positive.
I'm not opposed to merging this, but maybe we can wait for a third opinion.
Yes, to be clear:
I'm happy to close this PR if others feel similarly -- but I think your points also serve equally well as arguments against the existing "Did you mean?" suggestions we have for other diagnostics. Our JSON schema should mean that users get autocomplete suggestions for rule names (and AI agents should be able to parse the schema as well), and we'll be adding autocomplete for TypedDict keys at some point (astral-sh/ty#2189) |
|
I don't have strong opinions here. I feel like there are two questions:
Point 1 applies equally to all our "do you mean" suggestions, but point 2 does not. If the user types a rule name, it is 100% clear that they intended to type a rule name. If they type a bad TypedDict key, it is possible that they are trying to use a TypedDict in a looser way, but it is very likely that they meant to type a valid TypedDict key. When there is an unresolved import, in contrast, I would say we have fairly low confidence that what they really meant to do is import a similarly-named stdlib module. So on the whole, I think I would come down weakly on the side of not merging this. |
|
Hmm -- while I absolutely agree that the amount of handwritten code is shrinking fast, I do think there will still be quite a bit of handwritten code for some time to come, so I don't think we should give up on usability improvements on that basis. I'm not sure I even agree that usability improvements relative to other type checkers should be less of a priority than they were before. I think having superior UX to other type checkers will continue to be a valuable asset for a while yet. However, I agree with your point that we have ~no signal on how many users actually type out stdlib import typos locally. And I also agree that there's a fairly good chance that we'll give an incorrect suggestion given that third-party modules are not included in the list of available modules we feed into the Levenshtein algorithm. Given that, there's a good chance that this feature will just be adding noise to lots of diagnostics. So those are pretty good reasons to reject this PR. Thanks both for the reviews! |
|
To be clear, I don't intend for point 1 to apply more broadly to all "UX" considerations! For example, I think good detailed diagnostic messages are very useful to both humans and LLMs, and useful to humans regardless of their IDE. I think it's a more nuanced question of which UX features we should prioritize. And it's not even intended to say that "did you mean" suggestions are not useful! Just that we have a lot of high-priority things to do, and I think there are good reasons to think maybe that particular UX feature shouldn't be at the top of our list. |
Summary
With our new Levenshtein implementation that landed in #23291, we can give suggestions here that are generally pretty good
Test Plan
snapshots