Skip to content

[ty] Fix narrowing of nonlocal variables with conditional assignments#22966

Merged
charliermarsh merged 1 commit intomainfrom
charlie/narrowed
Feb 3, 2026
Merged

[ty] Fix narrowing of nonlocal variables with conditional assignments#22966
charliermarsh merged 1 commit intomainfrom
charlie/narrowed

Conversation

@charliermarsh
Copy link
Member

@charliermarsh charliermarsh commented Jan 30, 2026

Summary

Given:

def _(maybe_float: float | None, certain_int: int, flag: bool) -> None:
  if isinstance(maybe_float, int):
      return
  x = maybe_float
  def _() -> None:
      nonlocal x
      if flag:
          x = certain_int
      assert x is not None
      +x  # error: Unary operator `+` is not supported for operand of type `int | float | None`

With the guard (removed in this PR), assert x is not None would have no effect, since int | (float & ~int) is not assignable to (float & ~int) | None.

Closes astral-sh/ty#2649.

@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 30, 2026

Typing conformance results

No changes detected ✅

@charliermarsh charliermarsh added the ty Multi-file analysis & type inference label Jan 30, 2026
@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 30, 2026

mypy_primer results

Changes were detected when running on open source projects
graphql-core (https://github.com/graphql-python/graphql-core)
- src/graphql/validation/rules/stream_directive_on_list_field.py:42:38: warning[possibly-missing-attribute] Attribute `of_type` may be missing on object of type `GraphQLScalarType | GraphQLObjectType | GraphQLInterfaceType | ... omitted 4 union elements`
- Found 638 diagnostics
+ Found 637 diagnostics

pydantic (https://github.com/pydantic/pydantic)
- pydantic/_internal/_core_metadata.py:87:54: error[invalid-assignment] Invalid assignment to key "pydantic_js_extra" with declared type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | ((dict[str, Divergent], type[Any], /) -> None)` on TypedDict `CoreMetadata`: value of type `dict[object, object]`
+ pydantic/_internal/_core_metadata.py:87:54: error[invalid-assignment] Invalid assignment to key "pydantic_js_extra" with declared type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | ((dict[str, int | float | str | ... omitted 3 union elements], type[Any], /) -> None)` on TypedDict `CoreMetadata`: value of type `dict[object, object]`
- pydantic/fields.py:949:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:949:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:989:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:989:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1032:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:1032:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1072:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:1072:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1115:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:1115:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1154:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:1154:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1194:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:1194:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1573:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`, found `Top[dict[Unknown, Unknown]] | (((dict[str, Divergent], /) -> None) & ~Top[dict[Unknown, Unknown]]) | None`
+ pydantic/fields.py:1573:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`, found `Top[dict[Unknown, Unknown]] | (((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) & ~Top[dict[Unknown, Unknown]]) | None`

mypy (https://github.com/python/mypy)
- mypy/semanal_namedtuple.py:418:26: error[unresolved-attribute] Object of type `Expression` has no attribute `value`
- Found 1731 diagnostics
+ Found 1730 diagnostics

xarray (https://github.com/pydata/xarray)
+ xarray/computation/rolling.py:1216:20: error[invalid-return-type] Return type does not match returned value: expected `T_Xarray@Coarsen`, found `DataArray`
- xarray/computation/rolling.py:1192:19: error[invalid-argument-type] Argument to bound method `_to_temp_dataset` is incorrect: Expected `DataArray`, found `T_Xarray@Coarsen`
- xarray/computation/rolling.py:1216:20: error[invalid-argument-type] Argument to bound method `_from_temp_dataset` is incorrect: Argument type `T_Xarray@Coarsen` does not satisfy upper bound `DataArray` of type variable `Self`
- Found 1770 diagnostics
+ Found 1769 diagnostics

scikit-build-core (https://github.com/scikit-build/scikit-build-core)
+ src/scikit_build_core/build/wheel.py:99:20: error[no-matching-overload] No overload of bound method `__init__` matches arguments
- Found 46 diagnostics
+ Found 47 diagnostics

static-frame (https://github.com/static-frame/static-frame)
+ static_frame/core/bus.py:645:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[Bus[Any], object_]`, found `InterGetItemLocReduces[Bottom[Bus[Any]] | Bottom[Series[Any, Any]] | TypeBlocks | ... omitted 6 union elements, object_]`
- static_frame/core/bus.py:649:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Bus[Any], object_]`, found `InterGetItemILocReduces[Self@iloc, Self@iloc]`
+ static_frame/core/bus.py:649:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Bus[Any], object_]`, found `InterGetItemILocReduces[Bottom[Bus[Any]] | IndexHierarchy | TypeBlocks | ... omitted 7 union elements, Self@iloc]`
- static_frame/core/node_values.py:178:23: warning[possibly-missing-attribute] Attribute `index` may be missing on object of type `TVContainer_co@InterfaceValues`
+ static_frame/core/node_values.py:176:20: error[invalid-return-type] Return type does not match returned value: expected `TVContainer_co@InterfaceValues`, found `Top[Series[Any, Any]]`
+ static_frame/core/series.py:772:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Series[Any, Any], TVDtype@Series]`, found `InterGetItemILocReduces[Bottom[Series[Any, Any]] | IndexHierarchy | TypeBlocks | ... omitted 7 union elements, TVDtype@Series]`
+ static_frame/core/series.py:4072:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[SeriesHE[Any, Any], TVDtype@SeriesHE]`, found `InterGetItemILocReduces[Bottom[Series[Any, Any]] | IndexHierarchy | TypeBlocks | ... omitted 7 union elements, TVDtype@SeriesHE]`
+ static_frame/core/yarn.py:418:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Yarn[Any], object_]`, found `InterGetItemILocReduces[Bottom[Yarn[Any]] | Bottom[Index[Any]] | Bottom[Series[Any, Any]] | ... omitted 7 union elements, object_]`
- Found 1827 diagnostics
+ Found 1831 diagnostics

pandas (https://github.com/pandas-dev/pandas)
- pandas/core/arrays/datetimelike.py:2613:36: error[invalid-argument-type] Argument to function `py_get_unit_from_dtype` is incorrect: Expected `dtype[Any]`, found `ExtensionDtype`
- Found 3762 diagnostics
+ Found 3761 diagnostics

No memory usage changes detected ✅

// this can only happen if the code is unreachable
// and therefore it is correct to set the result to `Never`.
let union = union.build();
if union.is_assignable_to(db, ty) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mtshiba, I don't suppose you can remember the reason why this was added?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sharkdp may also know.

@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 30, 2026

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-return-type 3 4 4
invalid-argument-type 0 6 1
possibly-missing-attribute 0 5 1
invalid-assignment 0 4 1
unresolved-attribute 0 1 2
invalid-await 0 2 0
type-assertion-failure 2 0 0
no-matching-overload 0 1 0
unused-type-ignore-comment 1 0 0
Total 6 23 9

Full report with detailed diff (timing results)

@charliermarsh charliermarsh marked this pull request as ready for review January 30, 2026 02:15
@charliermarsh charliermarsh marked this pull request as draft January 30, 2026 02:16
@charliermarsh charliermarsh marked this pull request as ready for review January 30, 2026 02:23
@sharkdp sharkdp removed their request for review February 3, 2026 13:53
Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The removed guard makes the assumption that a narrowed type in a nested scope should never be wider than the type in the outer scope which binds the name. (Since we have nonlocal x in the nested scope, the outer scope here binds x.) This assumption seems reasonable, but is violated by the x = certain_int assignment in the nested scope here.

There's an open question here about how we should be reflecting the inner x = certain_int in the type of x in the outer scope. Because of nonlocal x, that assignment means that x can become int again in the outer scope, wherever the nested function is called. (Which in the simplified example in this test, is obviously "never"). I think probably the best balance here would be to consider that the definition of the nested scope itself should "apply" the inner nonlocal bindings in the outer scope, so anywhere in the outer scope after the def _() -> None: we treat x in the outer scope as could-be-int again. (The rationale here is that before the nested scope is defined, it can't possibly be called; after it is defined, it's not feasible to track every specific place where it might be called.) But all of this is a separate issue, and I don't think it removes the need for the change in this PR.

@carljm
Copy link
Contributor

carljm commented Feb 3, 2026

The ecosystem results all look correct here, with the exception of the new diagnostic in xarray -- but there we are triggering a pre-existing (and unrelated) issue where we narrow constrained typevars too eagerly.

@charliermarsh charliermarsh merged commit 848cb72 into main Feb 3, 2026
51 checks passed
@charliermarsh charliermarsh deleted the charlie/narrowed branch February 3, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem-analyzer ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nonlocal float | None variable asserted to not be None still has None in its type

3 participants