[ty] Use zip to perform explicit specialization#21635
Merged
dhruvmanila merged 7 commits intomainfrom Nov 27, 2025
Merged
Conversation
dhruvmanila
added
internal
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-11-27 03:49:07.980672923 +0000
+++ new-output.txt 2025-11-27 03:49:11.291679471 +0000
@@ -42,7 +42,7 @@
aliases_newtype.py:61:38: error[invalid-newtype] invalid base for `typing.NewType`: type `TD1`
aliases_newtype.py:63:15: error[invalid-newtype] Wrong number of arguments in `NewType` creation, expected 2, found 3
aliases_newtype.py:65:38: error[invalid-newtype] invalid base for `typing.NewType`: type `Any`
-aliases_type_statement.py:10:19: error[too-many-positional-arguments] Too many positional arguments: expected 1, got 3
+aliases_type_statement.py:10:35: error[invalid-type-arguments] Too many type arguments: expected 1, got 3
aliases_type_statement.py:17:1: error[unresolved-attribute] Object of type `typing.TypeAliasType` has no attribute `bit_count`
aliases_type_statement.py:19:1: error[call-non-callable] Object of type `TypeAliasType` is not callable
aliases_type_statement.py:23:7: error[unresolved-attribute] Object of type `typing.TypeAliasType` has no attribute `other_attrib`
@@ -63,13 +63,13 @@
aliases_type_statement.py:47:23: error[invalid-type-form] Int literals are not allowed in this context in a type expression
aliases_type_statement.py:48:23: error[invalid-type-form] Boolean operations are not allowed in type expressions
aliases_type_statement.py:49:23: error[fstring-type-annotation] Type expressions cannot use f-strings
-aliases_type_statement.py:75:81: error[too-many-positional-arguments] Too many positional arguments: expected 2, got 3
-aliases_type_statement.py:77:7: error[invalid-argument-type] Argument is incorrect: Expected `int`, found `str`
-aliases_type_statement.py:77:7: error[too-many-positional-arguments] Too many positional arguments: expected 2, got 3
-aliases_type_statement.py:78:7: error[too-many-positional-arguments] Too many positional arguments: expected 2, got 3
-aliases_type_statement.py:79:7: error[invalid-argument-type] Argument is incorrect: Expected `str`, found `int`
-aliases_type_statement.py:79:7: error[too-many-positional-arguments] Too many positional arguments: expected 2, got 3
-aliases_type_statement.py:80:7: error[too-many-positional-arguments] Too many positional arguments: expected 2, got 3
+aliases_type_statement.py:75:107: error[invalid-type-arguments] Too many type arguments: expected 2, got 3
+aliases_type_statement.py:77:27: error[invalid-type-arguments] Type `str` is not assignable to upper bound `int` of type variable `S@RecursiveTypeAlias2`
+aliases_type_statement.py:77:37: error[invalid-type-arguments] Too many type arguments: expected 2, got 3
+aliases_type_statement.py:78:37: error[invalid-type-arguments] Too many type arguments: expected 2, got 3
+aliases_type_statement.py:79:32: error[invalid-type-arguments] Type `int` is not assignable to upper bound `str` of type variable `T@RecursiveTypeAlias2`
+aliases_type_statement.py:79:37: error[invalid-type-arguments] Too many type arguments: expected 2, got 3
+aliases_type_statement.py:80:37: error[invalid-type-arguments] Too many type arguments: expected 2, got 3
aliases_type_statement.py:80:37: error[invalid-type-form] List literals are not allowed in this context in a type expression: Did you mean `tuple[int, str]`?
aliases_type_statement.py:82:1: error[cyclic-type-alias-definition] Cyclic definition of `RecursiveTypeAlias3`
aliases_type_statement.py:88:1: error[cyclic-type-alias-definition] Cyclic definition of `RecursiveTypeAlias6`
@@ -427,8 +427,8 @@
generics_base_class.py:29:14: error[invalid-type-form] `typing.Generic` is not allowed in type expressions
generics_base_class.py:30:8: error[invalid-type-form] `typing.Generic` is not allowed in type expressions
generics_base_class.py:45:5: error[type-assertion-failure] Type `Iterator[int]` does not match asserted type `Unknown`
-generics_base_class.py:49:22: error[too-many-positional-arguments] Too many positional arguments to class `LinkedList`: expected 1, got 2
-generics_base_class.py:61:18: error[too-many-positional-arguments] Too many positional arguments to class `MyDict`: expected 1, got 2
+generics_base_class.py:49:38: error[invalid-type-arguments] Too many type arguments to class `LinkedList`: expected 1, got 2
+generics_base_class.py:61:30: error[invalid-type-arguments] Too many type arguments to class `MyDict`: expected 1, got 2
generics_basic.py:34:12: error[unsupported-operator] Operator `+` is unsupported between objects of type `AnyStr@concat` and `AnyStr@concat`
generics_basic.py:49:44: error[invalid-legacy-type-variable] A `TypeVar` cannot have exactly one constraint
generics_basic.py:139:5: error[type-assertion-failure] Type `int` does not match asserted type `Unknown`
@@ -446,18 +446,18 @@
generics_defaults.py:38:1: error[type-assertion-failure] Type `type[OneDefault[int | float, bool]]` does not match asserted type `<class 'OneDefault[int | float, bool]'>`
generics_defaults.py:45:1: error[type-assertion-failure] Type `type[AllTheDefaults[Any, Any, str, int, bool]]` does not match asserted type `<class 'AllTheDefaults'>`
generics_defaults.py:46:1: error[type-assertion-failure] Type `type[AllTheDefaults[int, int | float | complex, str, int, bool]]` does not match asserted type `<class 'AllTheDefaults[int, int | float | complex, str, int, bool]'>`
-generics_defaults.py:50:1: error[missing-argument] No argument provided for required parameter `T2` of class `AllTheDefaults`
+generics_defaults.py:50:1: error[invalid-type-arguments] No type argument provided for required type variable `T2` of class `AllTheDefaults`
generics_defaults.py:52:1: error[type-assertion-failure] Type `type[AllTheDefaults[int, int | float | complex, str, int, bool]]` does not match asserted type `<class 'AllTheDefaults[int, int | float | complex, str, int, bool]'>`
generics_defaults.py:55:1: error[type-assertion-failure] Type `type[AllTheDefaults[int, int | float | complex, str, int, bool]]` does not match asserted type `<class 'AllTheDefaults[int, int | float | complex, str, int, bool]'>`
generics_defaults.py:59:1: error[type-assertion-failure] Type `type[AllTheDefaults[int, int | float | complex, str, int, bool]]` does not match asserted type `<class 'AllTheDefaults[int, int | float | complex, str, int, bool]'>`
generics_defaults.py:63:1: error[type-assertion-failure] Type `type[AllTheDefaults[int, int | float | complex, str, int, bool]]` does not match asserted type `<class 'AllTheDefaults[int, int | float | complex, str, int, bool]'>`
generics_defaults.py:79:1: error[type-assertion-failure] Type `Unknown` does not match asserted type `<class 'Class_ParamSpec'>`
-generics_defaults.py:79:35: error[too-many-positional-arguments] Too many positional arguments to class `Class_ParamSpec`: expected 1, got 2
+generics_defaults.py:79:56: error[invalid-type-arguments] Too many type arguments to class `Class_ParamSpec`: expected between 0 and 1, got 2
generics_defaults.py:80:1: error[type-assertion-failure] Type `Unknown` does not match asserted type `Class_ParamSpec[Unknown]`
-generics_defaults.py:80:32: error[too-many-positional-arguments] Too many positional arguments to class `Class_ParamSpec`: expected 1, got 2
+generics_defaults.py:80:53: error[invalid-type-arguments] Too many type arguments to class `Class_ParamSpec`: expected between 0 and 1, got 2
generics_defaults.py:81:1: error[type-assertion-failure] Type `Unknown` does not match asserted type `Class_ParamSpec[Unknown]`
generics_defaults.py:81:29: error[invalid-type-form] List literals are not allowed in this context in a type expression: Did you mean `tuple[bool, bool]`?
-generics_defaults.py:81:46: error[too-many-positional-arguments] Too many positional arguments to class `Class_ParamSpec`: expected 1, got 2
+generics_defaults.py:81:68: error[invalid-type-arguments] Too many type arguments to class `Class_ParamSpec`: expected between 0 and 1, got 2
generics_defaults.py:91:26: error[invalid-argument-type] `@Todo(starred expression)` is not a valid argument to `Generic`
generics_defaults.py:94:1: error[type-assertion-failure] Type `@Todo(specialized non-generic class)` does not match asserted type `<class 'Class_TypeVarTuple'>`
generics_defaults.py:95:1: error[type-assertion-failure] Type `@Todo(specialized non-generic class)` does not match asserted type `Class_TypeVarTuple`
@@ -505,7 +505,7 @@
generics_paramspec_specialization.py:40:27: error[invalid-type-form] List literals are not allowed in this context in a type expression: Did you mean `tuple[()]`?
generics_paramspec_specialization.py:40:31: error[invalid-type-form] List literals are not allowed in this context in a type expression: Did you mean `tuple[()]`?
generics_paramspec_specialization.py:52:22: error[invalid-type-form] List literals are not allowed in this context in a type expression: Did you mean `tuple[int, str, bool]`?
-generics_paramspec_specialization.py:58:15: error[too-many-positional-arguments] Too many positional arguments to class `ClassC`: expected 1, got 3
+generics_paramspec_specialization.py:58:27: error[invalid-type-arguments] Too many type arguments to class `ClassC`: expected 1, got 3
generics_scoping.py:14:1: error[type-assertion-failure] Type `int` does not match asserted type `Literal[1]`
generics_scoping.py:15:1: error[type-assertion-failure] Type `str` does not match asserted type `Literal["a"]`
generics_scoping.py:42:1: error[type-assertion-failure] Type `str` does not match asserted type `Literal["abc"]`
|
|
MichaReiser
reviewed
Nov 26, 2025
dhruvmanila
force-pushed
the
dhruv/zip-explicit-specialization
branch
from
8e10796 to
df59c94
Compare
dhruvmanila
commented
Nov 26, 2025
crates/ty_python_semantic/resources/mdtest/generics/legacy/classes.md
Outdated
Show resolved
Hide resolved
dhruvmanila
requested review from
AlexWaygood,
carljm,
dcreager and
sharkdp
as code owners
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
invalid-type-arguments |
437 | 0 | 0 |
too-many-positional-arguments |
0 | 288 | 0 |
invalid-argument-type |
0 | 105 | 2 |
missing-argument |
0 | 41 | 0 |
unused-ignore-comment |
3 | 0 | 0 |
| Total | 440 | 434 | 2 |
Member
Author
|
Hmm, I expected the diff numbers to be same, need to see where the difference is. |
Member
|
I remember that @sharkdp saw something similar in one of his past PRs and he added an explanation of when this can happen but I'm unable to find that PR right now |
Contributor
Yes, see: #21476 (comment). |
Member
Author
|
Oh, I see where the difference lies: The source code is: _ResamplerGroupBy: TypeAlias = (
DatetimeIndexResamplerGroupby[NDFrameT] # ty: ignore[invalid-argument-type]
| PeriodIndexResamplerGroupby[NDFrameT] # ty: ignore[invalid-argument-type]
| TimedeltaIndexResamplerGroupby[NDFrameT] # ty: ignore[invalid-argument-type]
)Here, the code has been changed from + pandas-stubs/core/groupby/groupby.pyi:74:35: error[invalid-type-arguments] Type `typing.TypeVar` is not assignable to upper bound `NDFrame` of type variable `NDFrameT@DatetimeIndexResamplerGroupby`
+ pandas-stubs/core/groupby/groupby.pyi:74:46: warning[unused-ignore-comment] Unused `ty: ignore` directive
+ pandas-stubs/core/groupby/groupby.pyi:75:35: error[invalid-type-arguments] Type `typing.TypeVar` is not assignable to upper bound `NDFrame` of type variable `NDFrameT@PeriodIndexResamplerGroupby`
+ pandas-stubs/core/groupby/groupby.pyi:75:46: warning[unused-ignore-comment] Unused `ty: ignore` directive
+ pandas-stubs/core/groupby/groupby.pyi:76:38: error[invalid-type-arguments] Type `typing.TypeVar` is not assignable to upper bound `NDFrame` of type variable `NDFrameT@TimedeltaIndexResamplerGroupby`
+ pandas-stubs/core/groupby/groupby.pyi:76:49: warning[unused-ignore-comment] Unused `ty: ignore` directiveThese are the additional 6 diagnostics. |
dcreager
approved these changes
Nov 26, 2025
Member
dcreager
left a comment
dcreager
left a comment
There was a problem hiding this comment.
This is great! I love that it finally lets us have more specific diagnostics.
| generic_context: GenericContext<'db>, | ||
| specialize: impl FnOnce(&[Option<Type<'db>>]) -> Type<'db>, | ||
| ) -> Type<'db> { | ||
| let db = self.db(); |
Member
There was a problem hiding this comment.
As a complete aside, I've come to the opinion that we should always be passing db as a parameter instead of storing it locally as a field. When we use the pattern here (saving it into a local variable), it's sometimes just to save typing from all of the places that we're doing self.db() (or self.db — another inconsistency!). But other times it's load bearing, and needed to satisfy the borrow checker! Committing to just always threading it through as a parameter would be more consistent and would eliminate the borrowck problems completely. The only places where we should store a db into a field is when we have to, to be able to implement some external trait.
But that's just me venting! No suggested change for this PR.
</end of soapbox>
| match item { | ||
| EitherOrBoth::Both(typevar, &provided_type) => { | ||
| if typevar.default_type(db).is_some() { | ||
| typevar_with_defaults += 1; |
Member
There was a problem hiding this comment.
This might be something you can instead track by taking the index if the first typevar with a default, since (I think) all of the defaulty typevars have to be grouped together at the end. For instance with
class C[T, U = int, V]: ...I don't think you can instantiate C with only two types (C[str, bool]). That would assume the bool lines up with U, and you haven't provided a type for V.
That said...I don't think that would be any simpler than what you've done here, so I would say leave it as is.
Member
Author
There was a problem hiding this comment.
Yeah, the typevar with default needs to be grouped together at the end of the list. I don't think we're raising any diagnostic for that (https://play.ty.dev/776bac63-9407-4ebf-acd7-d56d2c174f25) which is something that should be easy to do now. I can do it as a quick follow-up.
Member
Author
There was a problem hiding this comment.
Oh wait, this is a SyntaxError in CPython for PEP 695 type variables but it's a TypeError for the legacy type variables. I'll raise a ticket for now.
Member
Author
There was a problem hiding this comment.
- Use `Itertools::format` instead of `Itertools::join` - Use `Option::get_or_insert` instead of conditional insert
dhruvmanila
added
diagnostics
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR updates the explicit specialization logic to avoid using the call machinery.
Previously, the logic would use the call machinery by converting the list of type variables into a
Bindingwith a singleSignaturewhere all the type variables are positional-only parameters with bounds and constraints as the annotated type and the default type as the default parameter value. This has the advantage that it doesn't need to implement any specific logic but the disadvantages are subpar diagnostic messages as it would use the ones specific to a function call. But, an important disadvantage is that the kind of type variable is lost in this translation which becomes important in #21445 where aParamSpeccan specialize into a list of types which is provided using list literal. For example,This PR converts the logic to use a simple loop using
zip_longestas all type variables and their corresponding type argument maps on a 1-1 basis. They cannot be specified using keyword argument either e.g.,dict[_VT=str, _KT=int]is invalid.This PR also makes an initial attempt to improve the diagnostic message to specifically target the specialization part by using words like "type argument" instead of just "argument" and including information like the type variable, bounds, and constraints. Further improvements can be made by highlighting the type variable definition or the bounds / constraints as a sub-diagnostic but I'm going to leave that as a follow-up.
Test Plan
Update messages in existing test cases.