[ty] Add precise inference for indexing, slicing and unpacking NamedTuple instances#19560
Merged
AlexWaygood merged 7 commits intomainfrom Aug 13, 2025
Merged
[ty] Add precise inference for indexing, slicing and unpacking NamedTuple instances#19560AlexWaygood merged 7 commits intomainfrom
NamedTuple instances#19560AlexWaygood merged 7 commits intomainfrom
Conversation
AlexWaygood
force-pushed
the
alex/tuple-getitem
branch
from
aa99e4f to
1d950c9
Compare
AlexWaygood
force-pushed
the
alex-brent/namedtuples
branch
3 times, most recently
from
201d595 to
92bb82c
Compare
Contributor
|
This comment was marked as outdated.
This comment was marked as outdated.
Member
Author
|
The ecosystem report indicates that we shouldn't do this until we support unpacking tuple subclasses in the same way as we support unpacking tuples. |
AlexWaygood
force-pushed
the
alex/tuple-getitem
branch
2 times, most recently
from
e9b6681 to
7961ecd
Compare
AlexWaygood
force-pushed
the
alex-brent/namedtuples
branch
from
92bb82c to
c9282cb
Compare
Contributor
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-08-13 15:17:32.371829275 +0000
+++ new-output.txt 2025-08-13 15:17:32.438829393 +0000
@@ -1,5 +1,5 @@
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
-fatal[panic] Panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/918d35d/src/function/execute.rs:215:25 when checking `/home/runner/work/ruff/ruff/typing/conformance/tests/aliases_typealiastype.py`: `infer_definition_types(Id(c91d)): execute: too many cycle iterations`
+fatal[panic] Panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/918d35d/src/function/execute.rs:215:25 when checking `/home/runner/work/ruff/ruff/typing/conformance/tests/aliases_typealiastype.py`: `infer_definition_types(Id(14cb3)): execute: too many cycle iterations`
_directives_deprecated_library.py:15:31: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int`
_directives_deprecated_library.py:30:26: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `str`
_directives_deprecated_library.py:36:41: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__add__`
@@ -661,14 +661,8 @@
literals_semantics.py:24:5: error[invalid-assignment] Object of type `Literal[0]` is not assignable to `Literal[False]`
literals_semantics.py:25:5: error[invalid-assignment] Object of type `Literal[False]` is not assignable to `Literal[0]`
literals_semantics.py:33:5: error[invalid-assignment] Object of type `Literal[6, 7, 8]` is not assignable to `Literal[3, 4, 5]`
-namedtuples_define_class.py:23:1: error[type-assertion-failure] Argument does not have asserted type `int`
-namedtuples_define_class.py:24:1: error[type-assertion-failure] Argument does not have asserted type `int`
-namedtuples_define_class.py:25:1: error[type-assertion-failure] Argument does not have asserted type `str`
-namedtuples_define_class.py:26:1: error[type-assertion-failure] Argument does not have asserted type `str`
-namedtuples_define_class.py:27:1: error[type-assertion-failure] Argument does not have asserted type `int`
-namedtuples_define_class.py:28:1: error[type-assertion-failure] Argument does not have asserted type `int`
-namedtuples_define_class.py:29:1: error[type-assertion-failure] Argument does not have asserted type `tuple[int, int]`
-namedtuples_define_class.py:30:1: error[type-assertion-failure] Argument does not have asserted type `tuple[int, int, str]`
+namedtuples_define_class.py:32:7: error[index-out-of-bounds] Index 3 is out of bounds for tuple `Point` with length 3
+namedtuples_define_class.py:33:7: error[index-out-of-bounds] Index -4 is out of bounds for tuple `Point` with length 3
namedtuples_define_class.py:44:6: error[missing-argument] No argument provided for required parameter `y`
namedtuples_define_class.py:45:6: error[missing-argument] No argument provided for required parameter `y`
namedtuples_define_class.py:46:15: error[invalid-argument-type] Argument is incorrect: Expected `int`, found `Literal[""]`
@@ -679,15 +673,13 @@
namedtuples_define_class.py:95:1: error[type-assertion-failure] Argument does not have asserted type `int | float`
namedtuples_define_class.py:96:1: error[type-assertion-failure] Argument does not have asserted type `int | float`
namedtuples_define_class.py:98:19: error[invalid-argument-type] Argument is incorrect: Expected `str`, found `float`
-namedtuples_usage.py:27:1: error[type-assertion-failure] Argument does not have asserted type `int`
-namedtuples_usage.py:28:1: error[type-assertion-failure] Argument does not have asserted type `int`
-namedtuples_usage.py:29:1: error[type-assertion-failure] Argument does not have asserted type `str`
-namedtuples_usage.py:30:1: error[type-assertion-failure] Argument does not have asserted type `str`
-namedtuples_usage.py:31:1: error[type-assertion-failure] Argument does not have asserted type `int`
-namedtuples_usage.py:32:1: error[type-assertion-failure] Argument does not have asserted type `int`
+namedtuples_type_compat.py:22:1: error[invalid-assignment] Object of type `Point` is not assignable to `tuple[int, int]`
+namedtuples_type_compat.py:23:1: error[invalid-assignment] Object of type `Point` is not assignable to `tuple[int, str, str]`
+namedtuples_usage.py:34:7: error[index-out-of-bounds] Index 3 is out of bounds for tuple `Point` with length 3
+namedtuples_usage.py:35:7: error[index-out-of-bounds] Index -4 is out of bounds for tuple `Point` with length 3
namedtuples_usage.py:41:1: error[invalid-assignment] Cannot assign to object of type `Point` with no `__setitem__` method
-namedtuples_usage.py:49:1: error[type-assertion-failure] Argument does not have asserted type `int`
-namedtuples_usage.py:50:1: error[type-assertion-failure] Argument does not have asserted type `str`
+namedtuples_usage.py:52:1: error[invalid-assignment] Too many values to unpack: Expected 2
+namedtuples_usage.py:53:1: error[invalid-assignment] Not enough values to unpack: Expected 4
narrowing_typeguard.py:17:9: error[type-assertion-failure] Argument does not have asserted type `tuple[str, str]`
narrowing_typeguard.py:32:9: error[type-assertion-failure] Argument does not have asserted type `set[int]`
narrowing_typeguard.py:69:9: error[type-assertion-failure] Argument does not have asserted type `int`
@@ -868,5 +860,5 @@
typeddicts_operations.py:60:1: error[type-assertion-failure] Argument does not have asserted type `str | None`
typeddicts_type_consistency.py:101:1: error[invalid-assignment] Object of type `Unknown | None` is not assignable to `str`
typeddicts_usage.py:40:24: error[invalid-type-form] The special form `typing.TypedDict` is not allowed in type expressions. Did you mean to use a concrete TypedDict or `collections.abc.Mapping[str, object]` instead?
-Found 869 diagnostics
+Found 861 diagnostics
WARN A fatal error occurred while checking some files. Not all project files were analyzed. See the diagnostics list above for details. |
AlexWaygood
force-pushed
the
alex-brent/namedtuples
branch
from
c9282cb to
a79d285
Compare
AlexWaygood
force-pushed
the
alex-brent/namedtuples
branch
from
a79d285 to
b7e96a7
Compare
AlexWaygood
force-pushed
the
alex/remove-tuples
branch
8 times, most recently
from
3a8597e to
7ef83cd
Compare
AlexWaygood
force-pushed
the
alex-brent/namedtuples
branch
from
b7e96a7 to
32c5d8e
Compare
AlexWaygood
force-pushed
the
alex/remove-tuples
branch
2 times, most recently
from
7ad590d to
00b10f8
Compare
AlexWaygood
force-pushed
the
alex/remove-tuples
branch
from
255f90a to
7db1dd2
Compare
AlexWaygood
force-pushed
the
alex-brent/namedtuples
branch
from
e78cb63 to
a904ceb
Compare
AlexWaygood
force-pushed
the
alex/remove-tuples
branch
from
7db1dd2 to
299a0c5
Compare
AlexWaygood
force-pushed
the
alex-brent/namedtuples
branch
from
a904ceb to
e26f607
Compare
AlexWaygood
force-pushed
the
alex/remove-tuples
branch
from
299a0c5 to
23b108f
Compare
AlexWaygood
force-pushed
the
alex-brent/namedtuples
branch
from
e26f607 to
017ad8b
Compare
Member
Author
Ecosystem analysis
|
AlexWaygood
force-pushed
the
alex-brent/namedtuples
branch
from
d2f07ac to
6aaf82b
Compare
AlexWaygood
changed the title
NamedTuple instancesNamedTuple instances
carljm
approved these changes
Aug 12, 2025
|
|
||
| let pivot_class = ClassBase::try_from_type(db, pivot_class_type).ok_or({ | ||
| // TODO: having to get a class-literal just to pass it in here is silly. | ||
| // `BoundSuperType` should use a different enum rather than reusing `ClassBase`. |
Contributor
There was a problem hiding this comment.
I'm not convinced of this comment; I think the "pivot class" of a bound super object really is a "base class" (that is, it is anything that can exist in an MRO), and it's good to represent it with ClassBase.
I would frame the problem here slightly differently: ClassBase enum should represent "a thing that can be in an MRO", but the constructor ClassBase::try_from_type is also emulating the behavior of __mro_entries__, and there are cases for wanting to construct a "thing that can be in an MRO" from a type without emulating __mro_entries__. So perhaps there should be a way to construct a ClassBase that doesn't model __mro_entries__ (and would thus return None if you tried to use NamedTuple as a base type.)
Member
Author
There was a problem hiding this comment.
I would frame the problem here slightly differently:
ClassBaseenum should represent "a thing that can be in an MRO", but the constructorClassBase::try_from_typeis also emulating the behavior of__mro_entries__, and there are cases for wanting to construct a "thing that can be in an MRO" from a type without emulating__mro_entries__. So perhaps there should be a way to construct aClassBasethat doesn't model__mro_entries__(and would thus returnNoneif you tried to useNamedTupleas a base type.)
Hmm, that's an interesting way of thinking about it... although we don't really attempt to fully emulate __mro_entries__ in ClassBase::try_from_type. We only really try to emulate the parts of __mro_entries__ that consist of 1:1 replacements of one object for another. Some stdlib __mro_entries__ methods implement more complex behaviour where e.g. bases are conditionally appended to the end of the bases list -- we emulate that behaviour in
ruff/crates/ty_python_semantic/src/types/mro.rs
Lines 73 to 102 in e12747a
To me this sort-of goes to show even more that the methods on ClassBase have (in their current state, at least) been designed only with the internals of our MRO implementation in mind, and don't necessarily make sense if you try to reuse the enum for other purposes. One solution to this (the one this TODO comment currently implies) is just to create a separate enum with the same variants as ClassBase, which you can losslessly convert a ClassBase into. Since ClassBase is not a very big enum at all, I don't think this would lead to much code duplication.
I think BoundSuperType pretty clearly shouldn't be using ClassBase::try_from_type here, at the very least: it leads to obvious false negatives like this -- the super() call here fails at runtime, but we don't detect that, because typing.ChainMap is valid as an entry in a class's bases tuple, so the ClassBase::try_from_type call succeeds:
import typing
# revealed: <super: <class 'ChainMap[Unknown, Unknown]'>, Unknown>
reveal_type(super(typing.ChainMap, typing.ChainMap()))For now I'll just make the TODO comment more vague and try to clean this up in a followup.
AlexWaygood
force-pushed
the
alex-brent/namedtuples
branch
from
bf12cee to
7cdf408
Compare
dcreager
added a commit
that referenced
this pull request
Aug 13, 2025
* main: [ty] Add precise inference for indexing, slicing and unpacking `NamedTuple` instances (#19560) Add rule code to GitLab description (#19896) [ty] render docstrings in hover (#19882) [ty] simplify return type of place_from_declarations (#19884) [ty] Various minor cleanups to tuple internals (#19891) [ty] Improve `sys.version_info` special casing (#19894)
dcreager
added a commit
that referenced
this pull request
Aug 13, 2025
…aints * dcreager/inferrable: (65 commits) this was right after all mark typevars inferrable as we go fix tests fix inference of constrained typevars [ty] Add precise inference for indexing, slicing and unpacking `NamedTuple` instances (#19560) Add rule code to GitLab description (#19896) [ty] render docstrings in hover (#19882) [ty] simplify return type of place_from_declarations (#19884) [ty] Various minor cleanups to tuple internals (#19891) [ty] Improve `sys.version_info` special casing (#19894) Don't cache files with diagnostics (#19869) [ty] support recursive type aliases (#19805) [ty] Remove unsafe `salsa::Update` implementations in `tuple.rs` (#19880) [ty] Function argument inlay hints (#19269) [ty] Remove Salsa interning for `TypedDictType` (#19879) Fix `lint.future-annotations` link (#19876) [ty] Reduce memory usage of `TupleSpec` and `TupleType` (#19872) [ty] Track heap usage of salsa structs (#19790) Update salsa to pull in tracked struct changes (#19843) [ty] simplify CycleDetector::visit signature (#19873) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Infer the correct tuple supertype for
NamedTupleclasses. This allows us to infer precise types for indexing intoNamedTupleclasses, unpackingNamedTupleclasses, and comparingNamedTupleclasses.Test Plan
Mdtests.
Typing conformance diff
This is all great! We see lots of
type-assertion-failuresgoing away, and lots of diagnostics now being emitted on lines in the conformance tests that are meant to produce type-checker errors.Ecosystem analysis
This all looks good to me! Analysis in #19560 (comment).
New ecosystem panics
Unfortunately this causes many new ecosystem panics, because
packaginghas aNamedTupleclass that has a field annotated with a recursive type alias: https://github.com/pypa/packaging/blob/0055d4b8ff353455f0617690e609bc68a1f9ade2/src/packaging/_parser.py#L55. The initial support #19805 added for recursive types doesn't help us here, because it only adds support for explicit PEP-695 type aliases, whereaspackaging's type alias is an implicit one.Many projects use
packaging, so this unfortunately causes quite a big fallout...Co-authored-by: Brent Westbrook [email protected]