[ty] More precise type inference for dictionary literals#20523
Merged
ibraheemdev merged 2 commits intomainfrom Sep 24, 2025
Merged
[ty] More precise type inference for dictionary literals#20523ibraheemdev merged 2 commits intomainfrom
ibraheemdev merged 2 commits intomainfrom
Conversation
ibraheemdev
requested review from
AlexWaygood,
carljm,
dcreager and
sharkdp
as code owners
Contributor
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-09-24 20:21:35.472748409 +0000
+++ new-output.txt 2025-09-24 20:21:38.636768402 +0000
@@ -39,7 +39,7 @@
aliases_implicit.py:107:9: error[invalid-type-form] Variable of type `list[Unknown | <class 'int'> | <class 'str'>]` is not allowed in a type expression
aliases_implicit.py:108:9: error[invalid-type-form] Variable of type `tuple[tuple[<class 'int'>, <class 'str'>]]` is not allowed in a type expression
aliases_implicit.py:109:9: error[invalid-type-form] Variable of type `list[@Todo(list comprehension element type)]` is not allowed in a type expression
-aliases_implicit.py:110:9: error[invalid-type-form] Variable of type `dict[@Todo(dict literal key type), @Todo(dict literal value type)]` is not allowed in a type expression
+aliases_implicit.py:110:9: error[invalid-type-form] Variable of type `dict[Unknown | str, Unknown | str]` is not allowed in a type expression
aliases_implicit.py:114:9: error[invalid-type-form] Variable of type `Literal[3]` is not allowed in a type expression
aliases_implicit.py:115:10: error[invalid-type-form] Variable of type `Literal[True]` is not allowed in a type expression
aliases_implicit.py:116:10: error[invalid-type-form] Variable of type `Literal[1]` is not allowed in a type expression
@@ -133,7 +133,7 @@
classes_classvar.py:38:11: error[invalid-type-form] Type qualifier `typing.ClassVar` expected exactly 1 argument, got 2
classes_classvar.py:39:14: error[invalid-type-form] Int literals are not allowed in this context in a type expression
classes_classvar.py:40:14: error[unresolved-reference] Name `var` used when not defined
-classes_classvar.py:52:5: error[invalid-assignment] Object of type `dict[@Todo(dict literal key type), @Todo(dict literal value type)]` is not assignable to `list[str]`
+classes_classvar.py:52:5: error[invalid-assignment] Object of type `dict[Unknown, Unknown]` is not assignable to `list[str]`
classes_classvar.py:55:17: error[invalid-type-form] Type qualifier `typing.ClassVar` is not allowed in type expressions (only in annotation expressions)
classes_classvar.py:69:23: error[invalid-type-form] `ClassVar` is not allowed in function parameter annotations
classes_classvar.py:70:12: error[invalid-type-form] `ClassVar` annotations are only allowed in class-body scopes
@@ -843,7 +843,7 @@
tuples_type_form.py:15:1: error[invalid-assignment] Object of type `tuple[Literal[1], Literal[""]]` is not assignable to `tuple[int, int]`
tuples_type_form.py:25:1: error[invalid-assignment] Object of type `tuple[Literal[1]]` is not assignable to `tuple[()]`
tuples_type_form.py:36:1: error[invalid-assignment] Object of type `tuple[Literal[1], Literal[2], Literal[3], Literal[""]]` is not assignable to `tuple[int, ...]`
-typeddicts_operations.py:37:5: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `Movie` constructor
+typeddicts_operations.py:37:20: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `Movie` constructor
typeddicts_operations.py:62:1: error[unresolved-attribute] Type `MovieOptional` has no attribute `clear`
typeddicts_readonly.py:24:4: error[invalid-assignment] Cannot assign to key "members" on TypedDict `Band`: key is marked read-only
typeddicts_readonly.py:50:4: error[invalid-assignment] Cannot assign to key "title" on TypedDict `Movie1`: key is marked read-only
@@ -851,13 +851,14 @@
typeddicts_readonly.py:60:4: error[invalid-assignment] Cannot assign to key "title" on TypedDict `Movie2`: key is marked read-only
typeddicts_readonly.py:61:4: error[invalid-assignment] Cannot assign to key "year" on TypedDict `Movie2`: key is marked read-only
typeddicts_readonly_inheritance.py:36:4: error[invalid-assignment] Cannot assign to key "name" on TypedDict `Album2`: key is marked read-only
-typeddicts_readonly_inheritance.py:65:1: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `RequiredName` constructor
+typeddicts_readonly_inheritance.py:65:19: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `RequiredName` constructor
typeddicts_type_consistency.py:69:21: error[invalid-key] Invalid key access on TypedDict `A3`: Unknown key "y"
-typeddicts_type_consistency.py:101:1: error[invalid-assignment] Object of type `@Todo(dict literal value type) | None` is not assignable to `str`
+typeddicts_type_consistency.py:101:1: error[invalid-assignment] Object of type `Unknown | None` is not assignable to `str`
+typeddicts_type_consistency.py:126:56: error[invalid-argument-type] Invalid argument to key "inner_key" with declared type `str` on TypedDict `Inner1`: value of type `Literal[1]`
typeddicts_usage.py:23:7: error[invalid-key] Invalid key access on TypedDict `Movie`: Unknown key "director"
typeddicts_usage.py:24:17: error[invalid-assignment] Invalid assignment to key "year" with declared type `int` on TypedDict `Movie`: value of type `Literal["1982"]`
-typeddicts_usage.py:28:1: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `Movie` constructor
+typeddicts_usage.py:28:17: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `Movie` constructor
typeddicts_usage.py:28:18: error[invalid-key] Invalid key access on TypedDict `Movie`: Unknown key "title"
typeddicts_usage.py:40:24: error[invalid-type-form] The special form `typing.TypedDict` is not allowed in type expressions. Did you mean to use a concrete TypedDict or `collections.abc.Mapping[str, object]` instead?
-Found 860 diagnostics
+Found 861 diagnostics
WARN A fatal error occurred while checking some files. Not all project files were analyzed. See the diagnostics list above for details. |
CodSpeed WallTime Performance ReportMerging #20523 will degrade performances by 50.92%Comparing Summary
Benchmarks breakdown
|
CodSpeed Instrumentation Performance ReportMerging #20523 will not alter performanceComparing Summary
Footnotes
|
Member
Author
|
It looks like mypy-primer is timing out due to some more issue related to union normalization. I opened astral-sh/ty#1237 to track this issue more generally. |
Contributor
|
It looks like the conformance suite regressions are because we don't understand the function form for creating a |
Contributor
|
I think we should switch primer to run in release mode and see how much that helps here. IIRC running it in debug mode originally wasn't really due to needing debug info in the run, it was because at the time it was fast enough in debug mode that saving the build time made it faster overall. I doubt that's the case today. We have enough real perf issues to sort out, we don't need to be also blocked by issues that only show up in debug builds. |
carljm
reviewed
Sep 23, 2025
Contributor
carljm
left a comment
carljm
left a comment
There was a problem hiding this comment.
This looks great! Just need to get a mypy run so we can see what surfaces there.
|
|
||
| self.infer_collection_literal(items, tcx, KnownClass::Dict) | ||
| .unwrap_or_else(|| { | ||
| KnownClass::Set.to_specialized_instance(self.db(), [Type::unknown()]) |
Contributor
There was a problem hiding this comment.
This looks like a copy-paste error?
We should add a test that would catch this.
Member
Author
There was a problem hiding this comment.
I don't actually know if this case is ever hit, because the constraint solver only fails if an inferred type fails to meet the upper bound of a type variable.
ibraheemdev
force-pushed
the
ibraheem/bidirectional-inference
branch
from
f93043f to
4944379
Compare
Member
ibraheemdev
force-pushed
the
ibraheem/bidirectional-inference
branch
from
4944379 to
44be253
Compare
Contributor
|
Contributor
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
invalid-argument-type |
982 | 0 | 122 |
unknown-argument |
694 | 0 | 0 |
possibly-missing-attribute |
195 | 2 | 25 |
invalid-assignment |
119 | 0 | 43 |
possibly-missing-implicit-call |
153 | 0 | 7 |
non-subscriptable |
102 | 0 | 2 |
unsupported-operator |
71 | 1 | 13 |
unused-ignore-comment |
0 | 79 | 0 |
no-matching-overload |
52 | 0 | 0 |
invalid-return-type |
15 | 0 | 32 |
call-non-callable |
43 | 0 | 1 |
index-out-of-bounds |
15 | 0 | 0 |
too-many-positional-arguments |
12 | 0 | 0 |
not-iterable |
7 | 0 | 4 |
missing-argument |
9 | 0 | 0 |
unresolved-attribute |
2 | 2 | 2 |
type-assertion-failure |
0 | 1 | 0 |
| Total | 2,471 | 85 | 251 |
Member
Author
|
From a quick skim, a lot of the errors seem to be of the form: def f(a: int, b: str): ...
x = { "a": 1, "b": "2" }
# Expected `int` found `Unknown | int | str`
f(x["a"], x["b"])
# or
f(**x)There were similar errors in the list/set inference PR, but it seems like a more common pattern in the ecosystem for dictionaries. pyright again accepts this, as it doesn't even attempt to infer a union and falls back to |
Contributor
|
I can see the rationale for pyright's behavior. This is a case where we can emit a lot of false positives on perfectly working untyped code, just because we don't infer a sufficiently precise type for the dict literal (which is implicitly used as a heterogeneous I think for now we should probably go ahead with this as-is, but in the future in "gradual guarantee" mode we may want to emulate pyright for container literals that "look heterogeneous". |
carljm
approved these changes
Sep 24, 2025
Contributor
|
Filed astral-sh/ty#1248 for follow-up later. |
Member
Author
I think this is actually tricky, because we don't currently have infrastructure for any other functional type forms (e.g. I added a fallback to |
Contributor
I think the workaround you put in place is fine for this PR. I am curious what the trickiest part of functional Nothing actionable for this PR, I'm just not convinced that landing the |
ibraheemdev
commented
Sep 24, 2025
| &self.context, | ||
| typed_dict, | ||
| dict, | ||
| dict.into(), |
Member
Author
There was a problem hiding this comment.
We don't have access to the target binding anymore, so this changes diagnostics slightly. Before:
error[missing-typed-dict-key]: Missing required key 'name' in TypedDict `Movie` constructor
--> x.py:8:5
|
7 | def func1(variable_key: str):
8 | movie: Movie = {variable_key: "", "year": 1900}
| ^^^^^
After:
error[missing-typed-dict-key]: Missing required key 'name' in TypedDict `Movie` constructor
--> x.py:8:20
|
7 | def func1(variable_key: str):
8 | movie: Movie = {variable_key: "", "year": 1900}
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I think this makes sense, as it extends nicely to errors in nested TypedDict literals.
Member
Author
|
The performance regression on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends #20360 to dictionary literals. This also improves our
TypeDictsupport by passing through nested type context.