[ty] default-specialize class-literal types in assignment to generic-alias types by carljm · Pull Request #21883 · astral-sh/ruff

carljm · 2025-12-10T02:10:42Z

Fixes astral-sh/ty#1832, fixes astral-sh/ty#1513

Summary

A class object C (for which we infer an unspecialized ClassLiteral type) should always be assignable to the type type[C] (which is default-specialized, if C is generic). We already implemented this for most cases, but we missed the case of a generic final type, where we simplify type[C] to the GenericAlias type for the default specialization of C. So we also need to implement this assignability of generic ClassLiteral types as-if default-specialized.

Test Plan

Added mdtests that failed before this PR.

astral-sh-bot · 2025-12-10T02:12:42Z

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

astral-sh-bot · 2025-12-10T02:14:40Z

`mypy_primer` results

Changes were detected when running on open source projects

mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
- bson/son.py:43:31: error[invalid-assignment] Object of type `<class 'Pattern[str]'>` is not assignable to `<class 'Pattern[Any]'>`
- Found 462 diagnostics
+ Found 461 diagnostics

setuptools (https://github.com/pypa/setuptools)
- setuptools/_vendor/typing_extensions.py:3279:21: error[invalid-argument-type] Argument to bound method `register` is incorrect: Expected `<class 'memoryview[int]'>`, found `<class 'memoryview'>`
- Found 1289 diagnostics
+ Found 1288 diagnostics

trio (https://github.com/python-trio/trio)
- src/trio/_channel.py:250:16: error[invalid-argument-type] Argument to bound method `_create` is incorrect: Expected `<class 'MemorySendChannel[SendType@MemorySendChannel]'>`, found `<class 'MemorySendChannel'>`
- src/trio/_channel.py:402:16: error[invalid-argument-type] Argument to bound method `_create` is incorrect: Expected `<class 'MemoryReceiveChannel[ReceiveType@MemoryReceiveChannel]'>`, found `<class 'MemoryReceiveChannel'>`
- Found 495 diagnostics
+ Found 493 diagnostics

dd-trace-py (https://github.com/DataDog/dd-trace-py)
- ddtrace/internal/settings/symbol_db.py:25:9: error[invalid-argument-type] Argument to bound method `d` is incorrect: Expected `<class 'Pattern[Unknown]'>`, found `<class 'Pattern'>`
- Found 8471 diagnostics
+ Found 8470 diagnostics

egglog-python (https://github.com/egraphs-good/egglog-python)
- python/egglog/exp/any_expr.py:643:11: error[invalid-argument-type] Argument to function `converter` is incorrect: Expected `<class 'slice[Any, Any, Any]'>`, found `<class 'slice'>`
- python/egglog/exp/array_api.py:989:5: error[invalid-argument-type] Argument to function `converter` is incorrect: Expected `<class 'slice[Any, Any, Any]'>`, found `<class 'slice'>`
- Found 1492 diagnostics
+ Found 1490 diagnostics

No memory usage changes detected ✅

carljm · 2025-12-10T02:19:17Z

Looks like this fixes a few current false positives on the ecosystem, but the main point of this is to unblock type-of-cls on classmethods, where this starts to come up a lot more.

dhruvmanila

Can you also add similar tests for ParamSpec? I think that should just work but if not, you can add a TODO comment and I can look at it later:

from typing import final


@final
class Foo[**P]: ...


def expects_type_foo(f: type[Foo]): ...
def expects_type_foo_of_int(x: type[Foo[int]]): ...


expects_type_foo(Foo)  # ok
expects_type_foo_of_int(Foo[int])  # ok
expects_type_foo_of_int(Foo[str])  # not ok


@final
class FooDefault[**P = [int, str]]: ...


def expects_type_foo_default(f: type[FooDefault]): ...
def expects_type_foo_default_of_int(f: type[FooDefault[int]]): ...
def expects_type_foo_default_of_int_str(f: type[FooDefault[int, str]]): ...


expects_type_foo_default(FooDefault)  # ok
expects_type_foo_default(FooDefault[int, str])  # ok
expects_type_foo_default_of_int_str(FooDefault)  # ok

expects_type_foo_default(FooDefault[int])  # not ok
expects_type_foo_default_of_int(FooDefault[str])  # not ok
expects_type_foo_default_of_int_str(FooDefault[str, int])  # not ok

crates/ty_python_semantic/src/types.rs

crates/ty_python_semantic/resources/mdtest/type_of/generics.md

sharkdp · 2025-12-10T10:36:03Z

@dhruvmanila @AlexWaygood could one of you please review my changes here?

(@dhruvmanila, I noticed that some of the ParamSpec tests were correctly issuing errors before I added that new match branch, but I guess that was just an artifact? Something seems to be missing there for ParamSpec specifically)

astral-sh-bot · 2025-12-10T10:36:18Z

`ecosystem-analyzer` results

Lint rule	Added	Removed	Changed
`invalid-argument-type`	0	6	0
`invalid-assignment`	0	1	0
`unsupported-base`	0	1	0
Total	0	8	0

Full report with detailed diff (timing results)

AlexWaygood · 2025-12-10T10:41:40Z

Does this now also fix astral-sh/ty#1513? Or do we also need to make some changes to is_disjoint_from for that?

Are the new ecosystem hits on pandas expected?

sharkdp · 2025-12-10T10:43:13Z

Does this now also fix astral-sh/ty#1513? Or do we also need to make some changes to is_disjoint_from for that?

It looks like it does!

Are the new ecosystem hits on pandas expected?

Looking into it

crates/ty_python_semantic/src/types.rs

AlexWaygood · 2025-12-10T10:45:45Z

Does this now also fix astral-sh/ty#1513? Or do we also need to make some changes to is_disjoint_from for that?

It looks like it does!

Amazing! Is it worth adding some tests for that specifically?

AlexWaygood · 2025-12-10T10:48:17Z

Might want to double-check our alias<->class disjointness logic anyway, in case it's now inconsistent with our subtyping logic for these types

dhruvmanila · 2025-12-10T11:30:12Z

I noticed that some of the ParamSpec tests were correctly issuing errors before I added that new match branch, but I guess that was just an artifact? Something seems to be missing there for ParamSpec specifically)

I tried out this PR (without the new branch that you added) and it seems to be the following branch that's handling it:

            (Type::GenericAlias(alias), _) => ClassType::from(alias)
                .metaclass_instance_type(db)
                .has_relation_to_impl(
                    db,
                    target,
                    inferable,
                    relation,
                    relation_visitor,
                    disjointness_visitor,
                ),

I might have to spend some time here to understand if there's anything missing or not.

dhruvmanila · 2025-12-10T13:50:03Z

Ah, I think it's because the inferred variance of a ParamSpec is always Bivariant. We'll need to change this up to either use Invariant (which is what Pyright uses) or Covariant (which is that mypy uses) based on the following example:

from typing import final

class A: ...
class B(A): ...
class C(B): ...
class D: ...

@final
class Foo[**P]: ...

def expects_type_foo_of_b(x: type[Foo[B]]): ...

expects_type_foo_of_b(Foo[A])  # only Pyright error
expects_type_foo_of_b(Foo[B])  # ok
expects_type_foo_of_b(Foo[C])  # both Pyright and mypy error
expects_type_foo_of_b(Foo[D])  # both Pyright and mypy error

I can do it as a follow-up unless you want to add it in this PR.

…alias types

AlexWaygood

LGTM, though I'd still love to double-check #21883 (comment) (do the property tests pass on this branch?)

crates/ty_python_semantic/src/types.rs

codspeed-hq · 2025-12-10T14:18:39Z

CodSpeed Performance Report

Merging #21883 will not alter performance

_{Comparing cjm/unspecassign (1138855) with main (7bf50e7)}

Summary

✅ 22 untouched
⏩ 30 skipped¹

30 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

crates/ty_python_semantic/resources/mdtest/type_of/generics.md

sharkdp · 2025-12-10T14:34:46Z

LGTM, though I'd still love to double-check #21883 (comment) (do the property tests pass on this branch?)

Yes, sorry. This was not a "please review this again" push, but rather just a "I want to see if the pandas false positives are gone now" push. I didn't want to move Carl's PR to draft 😄

sharkdp · 2025-12-10T15:40:13Z

I'd still love to double-check #21883 (comment) (do the property tests pass on this branch?)

The property tests pass on this branch, which might just be because we don't have the necessary "ingredients" in the type pool. I added some tests for is_assignable_to (all of which pass), and some tests for is_disjoint_from (some of which fail, but they also fail on main). I tried for a while to implement the necessary disjointness relations, but didn't get there. I'd like to postpone this (I'll create a ticket).

AlexWaygood · 2025-12-10T15:44:27Z

I'd like to postpone this

Fine by me!

carljm

Looks good, thank you for the additional fixes!

crates/ty_python_semantic/resources/mdtest/type_of/generics.md

carljm · 2025-12-10T15:50:16Z

crates/ty_python_semantic/resources/mdtest/type_of/generics.md

+# Also OK, the specialized `C[int, str]` is assignable to `type[C[int, str]]`
+expects_type_c_of_int_and_str(C[int, str])
+
+# TODO: these should be errors


Do we know what we are lacking here? Is there an issue to track it?

@dhruvmanila said he'll look into it.

I think we should just add a use of P in the body of C, and then we will infer P as covariant (or invariant, depending what kind of use we add) and these should be fixed. That's what we do in the non-ParamSpec tests here already.

I didn't manage to add P in an invariant or covariant position, and adding it contravariantly does not fix the tests. After talking to Carl, we'll postpone this as well. I'll open an issue.

carljm · 2025-12-10T15:57:34Z

I think it's because the inferred variance of a ParamSpec is always Bivariant. We'll need to change this up to either use Invariant (which is what Pyright uses) or Covariant (which is that mypy uses)

I think inferring an "unused" ParamSpec type parameter (as in your example, and as in the current tests here) as bivariant is correct, according to our current variance inference logic. We do have an issue open to change this and always infer covariant for unused typevars: astral-sh/ty#1728. When we do this, it should naturally impact ParamSpec type parameters also, I think.

If we want to match pyright and always infer all ParamSpec as invariant, regardless of use, that would be a separate change specific to ParamSpec. But I'm not sure I see the rationale for that.

carljm added ty Multi-file analysis & type inference ecosystem-analyzer labels Dec 10, 2025

carljm mentioned this pull request Dec 10, 2025

[ty] Support implicit type of cls in signatures #21771

Merged

carljm marked this pull request as ready for review December 10, 2025 02:25

carljm requested review from AlexWaygood, dcreager and sharkdp as code owners December 10, 2025 02:25

carljm requested a review from ibraheemdev December 10, 2025 03:49

dhruvmanila approved these changes Dec 10, 2025

View reviewed changes

AlexWaygood approved these changes Dec 10, 2025

View reviewed changes

crates/ty_python_semantic/src/types.rs Outdated Show resolved Hide resolved

crates/ty_python_semantic/src/types.rs Outdated Show resolved Hide resolved

sharkdp reviewed Dec 10, 2025

View reviewed changes

crates/ty_python_semantic/resources/mdtest/type_of/generics.md Show resolved Hide resolved

sharkdp added ecosystem-analyzer and removed ecosystem-analyzer labels Dec 10, 2025

AlexWaygood reviewed Dec 10, 2025

View reviewed changes

crates/ty_python_semantic/src/types.rs Outdated Show resolved Hide resolved

crates/ty_python_semantic/src/types.rs Outdated Show resolved Hide resolved

sharkdp force-pushed the cjm/unspecassign branch from 76a47c9 to 573af3f Compare December 10, 2025 13:42

carljm and others added 3 commits December 10, 2025 14:59

[ty] default-specialize class-literal types in assignment to generic-…

0d9429c

…alias types

Update comment

ece01a1

Fix assignability of P[int] to type[P] and type[P[int]]

f0f6080

sharkdp force-pushed the cjm/unspecassign branch from 573af3f to 76245a3 Compare December 10, 2025 13:59

sharkdp added ecosystem-analyzer and removed ecosystem-analyzer labels Dec 10, 2025

AlexWaygood approved these changes Dec 10, 2025

View reviewed changes

crates/ty_python_semantic/src/types.rs Outdated Show resolved Hide resolved

AlexWaygood reviewed Dec 10, 2025

View reviewed changes

crates/ty_python_semantic/resources/mdtest/type_of/generics.md Outdated Show resolved Hide resolved

sharkdp added 2 commits December 10, 2025 15:33

Nits

0304c31

Add test for #1513

1fd9dc5

Move test

ab260ac

sharkdp force-pushed the cjm/unspecassign branch from 76245a3 to b84f253 Compare December 10, 2025 15:37

sharkdp force-pushed the cjm/unspecassign branch from b84f253 to 37ac3de Compare December 10, 2025 15:43

Add is_assignable_to/is_disjoint_from tests

d500d7e

sharkdp force-pushed the cjm/unspecassign branch from 37ac3de to d500d7e Compare December 10, 2025 15:48

carljm commented Dec 10, 2025

View reviewed changes

Two more tests

1138855

sharkdp merged commit 951766d into main Dec 10, 2025
41 checks passed

sharkdp deleted the cjm/unspecassign branch December 10, 2025 16:18

sharkdp mentioned this pull request Dec 10, 2025

Explore and possibly fix variance for ParamSpec astral-sh/ty#1843

Open

dhruvmanila mentioned this pull request Jan 20, 2026

[ty] Fix ParamSpec variance to enable equivalence checking in assert_type #22754

Closed

Comments

Conversation

carljm commented Dec 10, 2025 • edited by AlexWaygood Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

astral-sh-bot bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Diagnostic diff on typing conformance tests

Uh oh!

astral-sh-bot bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mypy_primer results

Uh oh!

carljm commented Dec 10, 2025

Uh oh!

dhruvmanila left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sharkdp commented Dec 10, 2025

Uh oh!

astral-sh-bot bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ecosystem-analyzer results

Uh oh!

AlexWaygood commented Dec 10, 2025

Uh oh!

sharkdp commented Dec 10, 2025

Uh oh!

Uh oh!

Uh oh!

AlexWaygood commented Dec 10, 2025

Uh oh!

AlexWaygood commented Dec 10, 2025

Uh oh!

dhruvmanila commented Dec 10, 2025

Uh oh!

dhruvmanila commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codspeed-hq bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging #21883 will not alter performance

Summary

Footnotes

Uh oh!

Uh oh!

sharkdp commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sharkdp commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood commented Dec 10, 2025

Uh oh!

carljm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

carljm Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

sharkdp Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

carljm Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sharkdp Dec 10, 2025

carljm commented Dec 10, 2025 •

edited by AlexWaygood

Loading

astral-sh-bot bot commented Dec 10, 2025 •

edited

Loading

astral-sh-bot bot commented Dec 10, 2025 •

edited

Loading

`mypy_primer` results

astral-sh-bot bot commented Dec 10, 2025 •

edited

Loading

`ecosystem-analyzer` results

dhruvmanila commented Dec 10, 2025 •

edited

Loading

codspeed-hq bot commented Dec 10, 2025 •

edited

Loading

sharkdp commented Dec 10, 2025 •

edited

Loading

sharkdp commented Dec 10, 2025 •

edited

Loading

carljm Dec 10, 2025 •

edited

Loading

carljm commented Dec 10, 2025 •

edited

Loading