[ty] Offer "Did you mean...?" suggestions for unresolved `from` imports and unresolved attributes by AlexWaygood · Pull Request #18705 · astral-sh/ruff

AlexWaygood · 2025-06-16T12:16:25Z

Summary

This PR (a collaboration with @ntBre) adds "Did you mean...?" suggestions for unresolved from imports and unresolved attributes. For example:

The implementation uses the Levenshtein distance algorithm to calculate which possible member of the type is a closest match to the unresolved member that triggered the diagnostic. This specific Levenshtein implementation is an almost exact 1:1 port of CPython's implementation here, and many of the tests in this PR are also ported from CPython's tests. You can see CPython's Levenshtein implementation in action if you access a nonexistent attribute or import in the REPL:

Python 3.13.1 (main, Jan  3 2025, 12:04:03) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import collections
>>> collections.dequee
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    collections.dequee
AttributeError: module 'collections' has no attribute 'dequee'. Did you mean: 'deque'?
>>> from asyncio import gett_event_lop
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    from asyncio import gett_event_lop
ImportError: cannot import name 'gett_event_lop' from 'asyncio' (/Users/alexw/.pyenv/versions/3.13.1/lib/python3.13/asyncio/__init__.py). Did you mean: 'get_event_loop'?
>>>

Our motivation for copying CPython's implementation almost exactly is that CPython has had this feature for several Python versions now, and during that time many bug reports have been filed regarding incorrect suggestions, which have since been fixed. This implementation is thus very well "battle-tested" by this point; we can say with a reasonable degree of confidence that it gives good suggestions for unresolved members in the Python context.

Test Plan

Unit tests in levenshtein.rs
Mdtest integration tests

Co-authored-by: Brent Westbrook [email protected]

AlexWaygood · 2025-06-16T12:23:16Z

crates/ty_python_semantic/src/types/all_members.rs

This module has been renamed from ide_support to all_members because it's no longer just used for autocompletions; it's now also used for diagnostic suggestions.

AlexWaygood · 2025-06-16T12:30:44Z

Mypy_primer is failing because this .unwrap() call is panicking when checking sympy:

ruff/crates/ty_python_semantic/src/types/ide_support.rs

Line 189 in 869d7bf

let name = instance_attribute.sub_segments()[0].as_member().unwrap();

Which seems like it is probably a pre-existing bug exposed by this PR (but we obviously won't be able to land this PR without fixing it!).

AlexWaygood · 2025-06-16T13:09:28Z

Minimal repro for the primer panic on this branch:

class Point:
    def orthogonal_direction(self):
        self[0].is_zero


def test_point(p2: Point):
    p2.y

AlexWaygood · 2025-06-16T13:15:23Z

And yes, it's easy to trigger the panic on main by adding this test:

diff --git a/crates/ty_ide/src/completion.rs b/crates/ty_ide/src/completion.rs
index a8b93080b4..9f0a013b31 100644
--- a/crates/ty_ide/src/completion.rs
+++ b/crates/ty_ide/src/completion.rs
@@ -1835,6 +1835,23 @@ f"{f.<CURSOR>
         test.assert_completions_include("method");
     }
 
+    #[test]
+    fn no_panic_for_attribute_table_that_contains_subscript() {
+        let test = cursor_test(
+            r#"
+class Point:
+    def orthogonal_direction(self):
+        self[0].is_zero
+
+def test_point(p2: Point):
+    p2.<CURSOR>
+"#,
+        );
+
+        let completions = test.completions();
+        assert_eq!(completions, "orthogonal_direction");
+    }

AlexWaygood · 2025-06-16T13:37:33Z

The primer panic should be fixed by #18707, but I won't attempt to rebase this PR branch on top of that as the commit history got somewhat messy here 😆

AlexWaygood · 2025-06-16T22:25:51Z

Mypy_primer is now timing out, but it appears that it ran ty successfully on all selected projects. I'm guessing it's taking ages to compute the diff between old_ty's results and new_ty's results, because there are many, many diagnostics where the message will change like so:

- error[unresolved-attribute] Type `Foo` has no attribute `bar`
+ error[unresolved-attribute] Type `Foo` has no attribute `bar`: Did you mean `baz`?

So this PR is ready for review!

carljm

I reviewed the tests and did a high-level review of the implementation.

This will probably make us look a lot slower on early benchmarks of codebases where we raise a lot of AttributeError false positives, but so it goes; we should eliminate the false positive AttributeErrors.

Very cool feature!

carljm · 2025-06-17T01:36:37Z

crates/ty_python_semantic/src/types/diagnostic/levenshtein.rs

+    }
+
+    // Filter out the unresolved member itself.
+    // Otherwise (due to our implementation of implicit instance attributes),


Not sure I understand this. It sounds like this is saying that we consider an access of self.attribute (no assignment) sufficient to include attribute in all_members? Do we really do that? Why?

It sounds like this is saying that we consider an access of self.attribute (no assignment) sufficient to include attribute in all_members? Do we really do that?

That does appear to be the case, yes. If I make this diff to the PR branch:

diff --git a/crates/ty_python_semantic/src/types/diagnostic/levenshtein.rs b/crates/ty_python_semantic/src/types/diagnostic/levenshtein.rs index 11fc59674e..ff1fc86a28 100644 --- a/crates/ty_python_semantic/src/types/diagnostic/levenshtein.rs +++ b/crates/ty_python_semantic/src/types/diagnostic/levenshtein.rs @@ -66,18 +66,6 @@ where return None; } - // Filter out the unresolved member itself. - // Otherwise (due to our implementation of implicit instance attributes), - // we end up giving bogus suggestions like this: - // - // ```python - // class Foo: - // _attribute = 42 - // def bar(self): - // print(self.attribute) # error: unresolved attribute `attribute`; did you mean `attribute`? - // ``` - let options = options.filter(|name| name != unresolved_member); -

then we get these changes to the snapshots:

And those seem to be self-evidently worse "Did you mean?" suggestions 😆

Why?

I'm not sure. I agree that it seems a bit suspect. @sharkdp, do you have any ideas off the top of your head for why attribute might be included in the list returned by all_members here? If not, I can look into this as a followup.

…m` imports and unresolved attributes (#18705)" This reverts commit 913f136.

…om` imports and unresolved attributes (#18705)" (#18721)

…ence * main: (71 commits) Bump 0.12.0 (astral-sh#18724) Revert "[ty] Offer "Did you mean...?" suggestions for unresolved `from` imports and unresolved attributes (astral-sh#18705)" (astral-sh#18721) [`flake8-return`] Stabilize only add `return None` at the end when fixing `implicit-return` (`RET503`) (astral-sh#18516) [`pyupgrade`] Stabilize `non-pep695-generic-function` (`UP047`) (astral-sh#18524) [`pyupgrade`] Stabilize `non-pep695-generic-class` (`UP046`) (astral-sh#18519) [`pandas-vet`] Deprecate `pandas-df-variable-name` (`PD901`) (astral-sh#18618) [`flake8-bandit`] Remove `suspicious-xmle-tree-usage` (`S320`) (astral-sh#18617) Stabilize `dataclass-enum` (`RUF049`) (astral-sh#18570) Stabilize `unnecessary-dict-index-lookup` (`PLR1733`) (astral-sh#18571) Remove rust-toolchain.toml from sdist (astral-sh#17925) Stabilize `starmap-zip` (`RUF058`) (astral-sh#18525) [`flake8-logging`] Stabilize `exc-info-outside-except-handler` (`LOG014`) (astral-sh#18517) [`pyupgrade`] Stabilize `non-pep604-annotation-optional` (`UP045`) and preview behavior for `non-pep604-annotation-union` (`UP007`) (astral-sh#18505) Stabilize `pytest-warns-too-broad` (`PT030`) (astral-sh#18568) Stabilize `for-loop-writes` (`FURB122`) (astral-sh#18565) Stabilize `pytest-warns-with-multiple-statements` (`PT031`) (astral-sh#18569) Stabilize `pytest-parameter-with-default-argument` (`PT028`) (astral-sh#18566) Stabilize `nan-comparison` (`PLW0177`) (astral-sh#18559) Stabilize `check-and-remove-from-set` (`FURB132`) (astral-sh#18560) Stabilize `unnecessary-round` (`RUF057`) (astral-sh#18563) ...

…lved `from` imports and unresolved attributes (#18705)" (#18721)" This reverts commit 685eac1.

…23291) ## Summary For a couple of diagnostics currently, we add a "Did you mean...?" diagnostic hint if it appears like there's an obvious typo that caused us to emit an error. The "Did you mean...?" suggestion is generated via the `strsim` Levenshtein implementation on `crates.io`. This PR replaces the `strsim` implementation of Levenshtein used to create these hints with a custom Levenshtein implementation based on the one that CPython itself uses to create these hints: ```pycon >>> class Foo: ... xyxy = 42 ... >>> Foo.xyxyz Traceback (most recent call last): File "<python-input-1>", line 1, in <module> Foo.xyxyz AttributeError: type object 'Foo' has no attribute 'xyxyz'. Did you mean: 'xyxy'? ``` The added tests are also derived from CPython's test suite. The motivation for copying CPython's implementation almost exactly is that CPython has had this feature for several Python versions now, and during that time many bug reports have been filed regarding incorrect suggestions, which have since been fixed. This implementation is thus very well "battle-tested" by this point; we can say with a reasonable degree of confidence that it gives good suggestions for typos in the Python context. The ecosystem report on this PR bears out that this is an improvement. We see bad suggestions going away: ```diff - [error] invalid-key - Unknown key "pair" for TypedDict `RPCAnalyzedDFMsg` - did you mean "data"? + [error] invalid-key - Unknown key "pair" for TypedDict `RPCAnalyzedDFMsg`: Unknown key "pair" ``` and good suggestions being added: ```diff - [error] invalid-key - Unknown key "old_entity_id" for TypedDict `_EventEntityRegistryUpdatedData_CreateRemove`: Unknown key "old_entity_id" + [error] invalid-key - Unknown key "old_entity_id" for TypedDict `_EventEntityRegistryUpdatedData_CreateRemove` - did you mean "entity_id"? ``` This Levenshtein implementation was originally proposed in #18705, and then again in #18751. Those PRs also made other changes to use the Levenshtein implementation in certain other areas, however, where computing the list of suggestions to pass into the Levenshtein algorithm turned out to be prohibitively expensive. This PR therefore _only_ updates the Levenshtein implementation being used for our existing subdiagnostics, rather than expanding the callsites of the Levenshtein implementation. ## Test plan Unit tests have been added in `levenshtein.rs`. Some mdtests and snapshots were updated to ensure that they still test what they're meant to be testing, even with the new Levenshtein implementation. Co-authored-by: Brent Westbrook <[email protected]>

AlexWaygood and others added 16 commits June 13, 2025 18:21

Basic implementation of Levenshtein

ed75004

more tests, attempt to port CPython's implementation

71c04cf

get everything compiling and module tests passing

305866c

accept snapshot that Python doesn't give a suggestion for either

c1e6488

accept fixed suggestion

0b668d7

add expected error annotation

f859ed0

Move to submodule and fix a few nits

b12cf86

Add suggestions for unresolved attributes too

dfe7353

Port more tests

2e0d3be

fix typos introduced when porting tests

f8291c2

Use test_case for other unit tests too

9bffbd1

use annotation message for both diagnostics

ab38f8f

minor cleanups

1188c55

More tests, fix tests, rename ide_support module

3ee7125

apply CPython's handling of suggestions that start with underscores

e0e7a4a

run pre-commit

a5adbaa

AlexWaygood requested review from carljm, dcreager and sharkdp as code owners June 16, 2025 12:16

AlexWaygood added ty Multi-file analysis & type inference diagnostics Related to reporting of diagnostics. labels Jun 16, 2025

AlexWaygood added 2 commits June 16, 2025 13:19

Merge branch 'main' into alex-brent/did-you-mean

4d357ea

update snapshots

ab16e70

AlexWaygood commented Jun 16, 2025

View reviewed changes

cargo dev generate-all

8b0b5bc

AlexWaygood requested a review from MichaReiser as a code owner June 16, 2025 12:28

fix checkout on Windows

f7cdb88

AlexWaygood force-pushed the alex-brent/did-you-mean branch from 76a50bf to f7cdb88 Compare June 16, 2025 12:52

AlexWaygood mentioned this pull request Jun 16, 2025

[ty] Fix panic when attempting to provide autocompletions for an instance of a class that assigns attributes to self[0] #18707

Merged

Merge branch 'main' into alex-brent/did-you-mean

067300e

carljm approved these changes Jun 17, 2025

View reviewed changes

Merge branch 'main' into alex-brent/did-you-mean

7633862

AlexWaygood merged commit 913f136 into main Jun 17, 2025
34 of 35 checks passed

AlexWaygood deleted the alex-brent/did-you-mean branch June 17, 2025 10:10

AlexWaygood added a commit that referenced this pull request Jun 17, 2025

Revert "[ty] Offer "Did you mean...?" suggestions for unresolved `fro…

ac35619

…m` imports and unresolved attributes (#18705)" This reverts commit 913f136.

AlexWaygood mentioned this pull request Jun 17, 2025

Revert "[ty] Offer "Did you mean...?" suggestions for unresolved from imports and unresolved attributes (#18705)" #18721

Merged

AlexWaygood added a commit that referenced this pull request Jun 17, 2025

Revert "[ty] Offer "Did you mean...?" suggestions for unresolved `fr…

685eac1

…om` imports and unresolved attributes (#18705)" (#18721)

AlexWaygood added a commit that referenced this pull request Jun 17, 2025

Revert " Revert "[ty] Offer "Did you mean...?" suggestions for unreso…

823f599

…lved `from` imports and unresolved attributes (#18705)" (#18721)" This reverts commit 685eac1.

AlexWaygood added a commit that referenced this pull request Jun 18, 2025

Revert " Revert "[ty] Offer "Did you mean...?" suggestions for unreso…

c55a3f7

…lved `from` imports and unresolved attributes (#18705)" (#18721)" This reverts commit 685eac1.

AlexWaygood mentioned this pull request Jun 18, 2025

Reapply "[ty] Offer "Did you mean...?" suggestions for unresolved from imports and unresolved attributes (#18705)" #18751

Draft

sharkdp mentioned this pull request Aug 5, 2025

[ty] Infer types for key-based access on TypedDicts #19763

Merged

AlexWaygood mentioned this pull request Dec 5, 2025

[ty] improve "Did you mean?" suggestion selection algorithm #21808

Closed

AlexWaygood mentioned this pull request Feb 15, 2026

[ty] Replace strsim with CPython-based Levenshtein implementation #23291

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

[ty] Offer "Did you mean...?" suggestions for unresolved `from` imports and unresolved attributes#18705

[ty] Offer "Did you mean...?" suggestions for unresolved `from` imports and unresolved attributes#18705
AlexWaygood merged 22 commits intomainfrom
alex-brent/did-you-mean

AlexWaygood commented Jun 16, 2025 •

edited

Loading

Uh oh!

AlexWaygood Jun 16, 2025

Uh oh!

AlexWaygood commented Jun 16, 2025 •

edited

Loading

Uh oh!

AlexWaygood commented Jun 16, 2025

Uh oh!

AlexWaygood commented Jun 16, 2025 •

edited

Loading

Uh oh!

AlexWaygood commented Jun 16, 2025

Uh oh!

AlexWaygood commented Jun 16, 2025

Uh oh!

carljm left a comment

Uh oh!

carljm Jun 17, 2025 •

edited by AlexWaygood

Loading

Uh oh!

AlexWaygood Jun 17, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Conversation

AlexWaygood commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

AlexWaygood Jun 16, 2025

Choose a reason for hiding this comment

Uh oh!

AlexWaygood commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood commented Jun 16, 2025

Uh oh!

AlexWaygood commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood commented Jun 16, 2025

Uh oh!

AlexWaygood commented Jun 16, 2025

Uh oh!

carljm left a comment

Choose a reason for hiding this comment

Uh oh!

carljm Jun 17, 2025 • edited by AlexWaygood Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlexWaygood Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AlexWaygood commented Jun 16, 2025 •

edited

Loading

AlexWaygood commented Jun 16, 2025 •

edited

Loading

AlexWaygood commented Jun 16, 2025 •

edited

Loading

carljm Jun 17, 2025 •

edited by AlexWaygood

Loading

AlexWaygood Jun 17, 2025 •

edited

Loading