Skip to content

[ty] fix comparisons and arithmetic with NewTypes of float#22105

Merged
oconnor663 merged 2 commits intomainfrom
newtype_float_ops
Jan 6, 2026
Merged

[ty] fix comparisons and arithmetic with NewTypes of float#22105
oconnor663 merged 2 commits intomainfrom
newtype_float_ops

Conversation

@oconnor663
Copy link
Contributor

@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 20, 2025

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 20, 2025

mypy_primer results

Changes were detected when running on open source projects
pydantic (https://github.com/pydantic/pydantic)
- pydantic/fields.py:943:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:943:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:983:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:983:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1026:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:1026:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1066:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:1066:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1109:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:1109:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1148:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:1148:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1188:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:1188:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1567:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`, found `Top[dict[Unknown, Unknown]] | (((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) & ~Top[dict[Unknown, Unknown]]) | None`
+ pydantic/fields.py:1567:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`, found `Top[dict[Unknown, Unknown]] | (((dict[str, Divergent], /) -> None) & ~Top[dict[Unknown, Unknown]]) | None`

xarray (https://github.com/pydata/xarray)
- xarray/core/dataarray.py:5757:16: error[invalid-return-type] Return type does not match returned value: expected `T_Xarray@map_blocks`, found `T_Xarray@map_blocks | DataArray | Dataset`
+ xarray/core/dataarray.py:5757:16: error[invalid-return-type] Return type does not match returned value: expected `T_Xarray@map_blocks`, found `DataArray | Dataset`
- xarray/core/dataset.py:8873:16: error[invalid-return-type] Return type does not match returned value: expected `T_Xarray@map_blocks`, found `T_Xarray@map_blocks | DataArray | Dataset`
+ xarray/core/dataset.py:8873:16: error[invalid-return-type] Return type does not match returned value: expected `T_Xarray@map_blocks`, found `DataArray | Dataset`

static-frame (https://github.com/static-frame/static-frame)
- static_frame/core/bus.py:671:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[Bus[Any], object_]`, found `InterGetItemLocReduces[Top[Bus[Any]] | TypeBlocks | Batch | ... omitted 6 union elements, object_]`
+ static_frame/core/bus.py:675:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Bus[Any], object_]`, found `InterGetItemILocReduces[Self@iloc | Bus[Any], object_ | Self@iloc]`
- static_frame/core/bus.py:675:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Bus[Any], object_]`, found `InterGetItemILocReduces[Top[Index[Any]] | TypeBlocks | Top[Bus[Any]] | ... omitted 6 union elements, generic[object]]`
- static_frame/core/series.py:772:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Series[Any, Any], TVDtype@Series]`, found `InterGetItemILocReduces[Top[Index[Any]] | TypeBlocks | Top[Bus[Any]] | ... omitted 6 union elements, generic[object]]`
+ static_frame/core/series.py:772:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Series[Any, Any], TVDtype@Series]`, found `InterGetItemILocReduces[Top[Index[Any]] | Top[Series[Any, Any]] | TypeBlocks | ... omitted 6 union elements, generic[object]]`
- static_frame/core/series.py:4072:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[SeriesHE[Any, Any], TVDtype@SeriesHE]`, found `InterGetItemILocReduces[Top[Index[Any]] | TypeBlocks | Top[Bus[Any]] | ... omitted 7 union elements, generic[object]]`
+ static_frame/core/series.py:4072:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[SeriesHE[Any, Any], TVDtype@SeriesHE]`, found `InterGetItemILocReduces[TypeBlocks | Batch | SeriesAssign | ... omitted 6 union elements, generic[object]]`
- static_frame/core/yarn.py:418:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Yarn[Any], object_]`, found `InterGetItemILocReduces[Top[Yarn[Any]] | TypeBlocks | Batch | ... omitted 6 union elements, generic[object]]`
+ static_frame/core/yarn.py:418:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Yarn[Any], object_]`, found `InterGetItemILocReduces[Self@iloc | Yarn[Any], generic[object]]`
- Found 1842 diagnostics
+ Found 1841 diagnostics

No memory usage changes detected ✅

@AlexWaygood AlexWaygood added ty Multi-file analysis & type inference bug Something isn't working labels Dec 20, 2025
@oconnor663 oconnor663 force-pushed the newtype_float_ops branch 2 times, most recently from a2bb4c4 to 9cefff0 Compare December 30, 2025 01:50
@oconnor663
Copy link
Contributor Author

oconnor663 commented Dec 30, 2025

I've added support/tests for has_relation_to_impl, which was also evaluating assignability incorrectly in these cases, though that (surprisingly) didn't turn out to be the root cause for the original "unsupported operation" errors. I've also added a bunch of comments explaining why these new special cases are needed. This PR is now ready for review.

Edit: ChatGPT caught a real mistake below, which I think is responsible for the failing fuzz tests and the excessive new ecosystem errors. Will update.

@oconnor663 oconnor663 marked this pull request as ready for review December 30, 2025 01:51
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 30, 2025

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-await 0 2 6
invalid-return-type 0 1 5
possibly-missing-attribute 0 3 1
unresolved-attribute 0 0 2
invalid-argument-type 0 1 0
unused-ignore-comment 1 0 0
Total 1 7 14

Full report with detailed diff (timing results)

@oconnor663
Copy link
Contributor Author

Most of the new diagnostics are gone, but there are still a few that look like they're probably my fault:
image
Looking into it...

@oconnor663
Copy link
Contributor Author

Minimal repro for these failures, new in this PR:

from typing import NewType

Key = NewType("Key", object)

class ThingWithKeys:
    def __contains__(self, refname: Key) -> bool:
        return False

def _(key: Key, keys: ThingWithKeys):
    key in keys  # error[unsupported-operator]: Unsupported `in` operation

@oconnor663
Copy link
Contributor Author

Ok all of the added and removed diagnostics are gone. (Is there really nobody in the ecosystem report trying to do arithmetic on newtypes of float?) What remains is just unions that get printed differently. Some of them look like random ordering flakes, but others are cases where int and float simplify out now, which seems reasonable.

@oconnor663
Copy link
Contributor Author

94d22ea is a big rebase on top of #22232.

Copy link
Contributor

@Gankra Gankra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all makes sense on paper. This special-case is only triggered by float/complex because NewType otherwise doesn't permit taking a union as an argument, right? Are there other cases where you can smuggle unions that we might not be checking for though? Type aliases (of various flavours)?

Comment on lines 324 to 330
reveal_type(Foo(3.14) + Bing()) # revealed: Foo
reveal_type(Bing() + Foo(42)) # revealed: Foo
reveal_type(Foo(3.14) < Bing()) # revealed: bool
reveal_type(Bing() < Foo(42)) # revealed: bool
reveal_type(Foo(3.14) in Bing()) # revealed: bool
```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you include the negative cases like you had for int? Those seemed important/valuable.

@oconnor663
Copy link
Contributor Author

This special-case is only triggered by float/complex because NewType otherwise doesn't permit taking a union as an argument, right?

Right. That restriction is implemented here. Apart from the float and int special cases, only class types and other newtypes are allowed in the base.

@oconnor663 oconnor663 merged commit ab1ac25 into main Jan 6, 2026
48 checks passed
@oconnor663 oconnor663 deleted the newtype_float_ops branch January 6, 2026 17:32
carljm added a commit that referenced this pull request Jan 13, 2026
…king

When calling a method on an instance of a generic class with bounded type
parameters (e.g., `C[T: K]` where `K` is a NewType), ty was incorrectly reporting:
"Argument type `C[K]` does not satisfy upper bound `C[T@C]` of type variable `Self`"

The issue was introduced by PR #22105, which added a catch-all case for NewType
assignments that falls back to the concrete base type. This case was placed
before the TypeVar handling cases, so when checking `K <: T@C` (where K is a
NewType and T@C is a TypeVar with upper bound K):

1. The NewType fallback matched first
2. It delegated to `int` (K's concrete base type)
3. Then checked `int <: T@C`, which checks if `int` satisfies bound `K`
4. But `int` is not assignable to `K` (NewTypes are distinct from their bases)

The fix moves the NewType fallback case after the TypeVar cases, so TypeVar
handling takes precedence. Now when checking `K <: T@C`, we use the TypeVar
case at line 828 which returns `false` for non-inferable typevars - but this
is correct because the *other* direction (`T@C <: K`) passes, and for the
overall specialization comparison both directions are checked.

Fixes astral-sh/ty#2467

Co-Authored-By: Carl Meyer <[email protected]>
carljm added a commit that referenced this pull request Jan 13, 2026
…king

When calling a method on an instance of a generic class with bounded type
parameters (e.g., `C[T: K]` where `K` is a NewType), ty was incorrectly reporting:
"Argument type `C[K]` does not satisfy upper bound `C[T@C]` of type variable `Self`"

The issue was introduced by PR #22105, which added a catch-all case for NewType
assignments that falls back to the concrete base type. This case was placed
before the TypeVar handling cases, so when checking `K <: T@C` (where K is a
NewType and T@C is a TypeVar with upper bound K):

1. The NewType fallback matched first
2. It delegated to `int` (K's concrete base type)
3. Then checked `int <: T@C`, which checks if `int` satisfies bound `K`
4. But `int` is not assignable to `K` (NewTypes are distinct from their bases)

The fix moves the NewType fallback case after the TypeVar cases, so TypeVar
handling takes precedence. Now when checking `K <: T@C`, we use the TypeVar
case at line 828 which returns `false` for non-inferable typevars - but this
is correct because the *other* direction (`T@C <: K`) passes, and for the
overall specialization comparison both directions are checked.

Fixes astral-sh/ty#2467

Co-Authored-By: Carl Meyer <[email protected]>
carljm added a commit that referenced this pull request Jan 13, 2026
…king

When calling a method on an instance of a generic class with bounded
type parameters (e.g., `C[T: K]` where `K` is a NewType), ty was
incorrectly reporting: "Argument type `C[K]` does not satisfy upper
bound `C[T@C]` of type variable `Self`"

The issue was introduced by PR #22105, which moved the catch-all case
for NewType assignments that falls back to the concrete base type. This
case was placed before the TypeVar handling cases, so when checking `K
<: T@C` (where K is a NewType and T@C is a TypeVar with upper bound K):

1. The NewType fallback matched first
2. It delegated to `int` (K's concrete base type)
3. Then checked `int <: T@C`, which checks if `int` satisfies bound `K`
4. But `int` is not assignable to `K` (NewTypes are distinct from their bases)

The fix moves the NewType fallback case after the TypeVar cases, so TypeVar
handling takes precedence.

Fixes astral-sh/ty#2467
carljm added a commit that referenced this pull request Jan 13, 2026
)

Fixes astral-sh/ty#2467

When calling a method on an instance of a generic class with bounded
type parameters (e.g., `C[T: K]` where `K` is a NewType), ty was
incorrectly reporting: "Argument type `C[K]` does not satisfy upper
bound `C[T@C]` of type variable `Self`"

The issue was introduced by PR #22105, which moved the catch-all case
for NewType assignments that falls back to the concrete base type. This
case was moved before the TypeVar handling cases, so when checking `K <:
T@C` (where K is a NewType and T@C is a TypeVar with upper bound K):

1. The NewType fallback matched first
2. It delegated to `int` (K's concrete base type)
3. Then checked `int <: T@C`, which checks if `int` satisfies bound `K`
4. But `int` is not assignable to `K` (NewTypes are distinct from their
bases)

The fix moves the NewType fallback case after the TypeVar cases, so
TypeVar handling takes precedence. Now when checking `K <: T@C`, we use
the TypeVar case at line 828 which returns `false` for non-inferable
typevars - but this is correct because the *other* direction (`T@C <:
K`) passes, and for the overall specialization comparison both
directions are checked.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ecosystem-analyzer ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NewTypes with base float do not inherit operators correctly

3 participants