Skip to content

[ty] Right-hand side narrowing for if Foo is type(x) expressions#22608

Merged
AlexWaygood merged 2 commits intomainfrom
alex/rhs-type-narrow
Jan 17, 2026
Merged

[ty] Right-hand side narrowing for if Foo is type(x) expressions#22608
AlexWaygood merged 2 commits intomainfrom
alex/rhs-type-narrow

Conversation

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Jan 15, 2026

Summary

This PR reworks the big match statement here into two let chains that are independently evaluated. This allows us to apply type narrowing on many expressions where we weren't able to before, including:

  • Right-hand-side narrowing for if Y is type(x)
  • Right-hand side narrowing for if Y is not type(x)
  • Right-hand side narrowing for <CALL EXPRESSION> is x where <CALL_EXPRESSION> is not a call to builtins.type
  • Right-hand side narrowing for <CALL EXPRESSION> is not x where <CALL_EXPRESSION> is not a call to builtins.type

The last two in particular are quite interesting: the AST for if type(x) is Y is very similar syntactically to if foo() is z, but we want to apply different kinds of narrowing for the two constructs:

  • For the former, we want to apply left-hand side narrowing for x, intersecting x's existing type with the top materialization of an instance of Y
  • For the latter, we want to apply right-hand side narrowing for z, intersecting z's type with the annotated return type of foo.

This is the key reason why a match no longer serves us well here: match arms are mutually exclusive! But we don't necessarily know which kind of narrowing we need to apply here until we've examined some of the types, so mutually exclusive branching is not what we want. We only want to continue if we succeeded in applying a narrowing constraint; otherwise, we want to fall through and check whether the other kind of narrowing would apply.

This PR is inspired by #22511, and seems to result in a surprising number of false positives going away in the ecosystem.

Test Plan

mdtests

@AlexWaygood AlexWaygood added the ty Multi-file analysis & type inference label Jan 15, 2026
@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 15, 2026

Typing conformance results

No changes detected ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 15, 2026

mypy_primer results

Changes were detected when running on open source projects
xarray (https://github.com/pydata/xarray)
- xarray/tests/test_variable.py:2753:53: warning[possibly-missing-attribute] Attribute `dtype` may be missing on object of type `Unknown | list[Unknown | list[Unknown | int]] | DataFrame`
- Found 1763 diagnostics
+ Found 1762 diagnostics

prefect (https://github.com/PrefectHQ/prefect)
- src/integrations/prefect-dbt/prefect_dbt/core/settings.py:94:28: error[invalid-assignment] Object of type `T@resolve_block_document_references | int | dict[str, Any] | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
+ src/integrations/prefect-dbt/prefect_dbt/core/settings.py:94:28: error[invalid-assignment] Object of type `T@resolve_block_document_references | dict[str, Any] | str | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
- src/integrations/prefect-dbt/prefect_dbt/core/settings.py:99:28: error[invalid-assignment] Object of type `int | T@resolve_variables | float | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
+ src/integrations/prefect-dbt/prefect_dbt/core/settings.py:99:28: error[invalid-assignment] Object of type `T@resolve_variables | str | int | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
- src/prefect/cli/deploy/_core.py:86:21: error[invalid-assignment] Object of type `T@resolve_block_document_references | int | dict[str, Any] | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
+ src/prefect/cli/deploy/_core.py:86:21: error[invalid-assignment] Object of type `T@resolve_block_document_references | dict[str, Any] | str | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
- src/prefect/cli/deploy/_core.py:87:21: error[invalid-assignment] Object of type `int | T@resolve_variables | float | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
+ src/prefect/cli/deploy/_core.py:87:21: error[invalid-assignment] Object of type `T@resolve_variables | str | int | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
- src/prefect/deployments/steps/core.py:137:38: error[invalid-argument-type] Argument is incorrect: Expected `T@resolve_variables`, found `T@resolve_block_document_references | int | dict[str, Any] | ... omitted 4 union elements`
+ src/prefect/deployments/steps/core.py:137:38: error[invalid-argument-type] Argument is incorrect: Expected `T@resolve_variables`, found `T@resolve_block_document_references | dict[str, Any] | str | ... omitted 4 union elements`
- src/prefect/utilities/templating.py:320:13: error[invalid-assignment] Invalid subscript assignment with key of type `object` and value of type `T@resolve_block_document_references | int | dict[str, Any] | ... omitted 4 union elements` on object of type `dict[str, Any]`
+ src/prefect/utilities/templating.py:320:13: error[invalid-assignment] Invalid subscript assignment with key of type `object` and value of type `T@resolve_block_document_references | dict[str, Any] | str | ... omitted 4 union elements` on object of type `dict[str, Any]`
- src/prefect/utilities/templating.py:323:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_block_document_references | dict[str, Any]`, found `list[T@resolve_block_document_references | int | dict[str, Any] | ... omitted 5 union elements]`
+ src/prefect/utilities/templating.py:323:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_block_document_references | dict[str, Any]`, found `list[T@resolve_block_document_references | dict[str, Any] | str | ... omitted 5 union elements]`
- src/prefect/utilities/templating.py:437:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `dict[object, int | T@resolve_variables | float | ... omitted 5 union elements]`
+ src/prefect/utilities/templating.py:437:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `dict[object, T@resolve_variables | str | int | ... omitted 5 union elements]`
- src/prefect/utilities/templating.py:442:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `list[int | T@resolve_variables | float | ... omitted 5 union elements]`
+ src/prefect/utilities/templating.py:442:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `list[T@resolve_variables | str | int | ... omitted 5 union elements]`
- src/prefect/workers/base.py:232:13: error[invalid-argument-type] Argument is incorrect: Expected `T@resolve_variables`, found `T@resolve_block_document_references | int | dict[str, Any] | ... omitted 4 union elements`
+ src/prefect/workers/base.py:232:13: error[invalid-argument-type] Argument is incorrect: Expected `T@resolve_variables`, found `T@resolve_block_document_references | dict[str, Any] | str | ... omitted 4 union elements`
- src/prefect/workers/base.py:234:20: error[invalid-argument-type] Argument expression after ** must be a mapping type: Found `int | T@resolve_variables | float | ... omitted 4 union elements`
+ src/prefect/workers/base.py:234:20: error[invalid-argument-type] Argument expression after ** must be a mapping type: Found `T@resolve_variables | str | int | ... omitted 4 union elements`

pandas (https://github.com/pandas-dev/pandas)
- pandas/tests/scalar/test_na_scalar.py:249:12: error[no-matching-overload] No overload matches arguments
- pandas/tests/scalar/test_na_scalar.py:250:14: error[no-matching-overload] No overload matches arguments
- pandas/tests/scalar/test_na_scalar.py:253:14: error[no-matching-overload] No overload matches arguments
- pandas/tests/scalar/test_na_scalar.py:275:14: error[no-matching-overload] No overload matches arguments
- Found 3756 diagnostics
+ Found 3752 diagnostics

sympy (https://github.com/sympy/sympy)
- sympy/functions/elementary/tests/test_complexes.py:43:12: error[no-matching-overload] No overload of function `__new__` matches arguments
- sympy/functions/elementary/tests/test_complexes.py:45:12: error[no-matching-overload] No overload of function `__new__` matches arguments
- sympy/functions/elementary/tests/test_complexes.py:46:12: error[no-matching-overload] No overload of function `__new__` matches arguments
- sympy/functions/elementary/tests/test_complexes.py:211:12: error[unresolved-attribute] Object of type `im` has no attribute `as_immutable`
- sympy/functions/elementary/tests/test_exponential.py:623:12: error[unresolved-attribute] Object of type `Expr` has no attribute `epsilon_eq`
- sympy/functions/elementary/tests/test_exponential.py:625:12: error[unresolved-attribute] Object of type `Expr` has no attribute `epsilon_eq`
- Found 15627 diagnostics
+ Found 15621 diagnostics

static-frame (https://github.com/static-frame/static-frame)
- static_frame/core/bus.py:671:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[Bus[Any], object_]`, found `InterGetItemLocReduces[Bus[Any] | Bottom[Index[Any]] | Bottom[Series[Any, Any]] | ... omitted 6 union elements, object_]`
- static_frame/core/bus.py:675:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Bus[Any], object_]`, found `InterGetItemILocReduces[Bus[Any] | Bottom[Index[Any]] | TypeBlocks | ... omitted 6 union elements, object_ | Self@iloc]`
+ static_frame/core/bus.py:675:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Bus[Any], object_]`, found `InterGetItemILocReduces[Self@iloc | Bus[Any], object_ | Self@iloc]`
- static_frame/core/node_selector.py:526:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[TVContainer_co@InterfaceSelectQuartet, Any]`, found `InterGetItemLocReduces[Bottom[Series[Any, Any]] | Unknown, Any]`
+ static_frame/core/node_selector.py:526:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[TVContainer_co@InterfaceSelectQuartet, Any]`, found `InterGetItemLocReduces[Unknown | Bottom[Series[Any, Any]], Any]`
+ static_frame/core/series.py:772:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Series[Any, Any], TVDtype@Series]`, found `InterGetItemILocReduces[Series[Any, Any] | Bottom[Index[Any]] | ndarray[Never, Never] | ... omitted 6 union elements, TVDtype@Series]`
- static_frame/core/series.py:4072:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[SeriesHE[Any, Any], TVDtype@SeriesHE]`, found `InterGetItemILocReduces[Bottom[Series[Any, Any]] | Bottom[Index[Any]] | TypeBlocks | ... omitted 7 union elements, TVDtype@SeriesHE]`
- static_frame/core/yarn.py:418:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Yarn[Any], object_]`, found `InterGetItemILocReduces[Yarn[Any] | ndarray[Never, Never] | TypeBlocks | ... omitted 6 union elements, object_]`
- Found 1824 diagnostics
+ Found 1822 diagnostics

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
- tests/frame/test_groupby.py:229:15: error[type-assertion-failure] Type `Series[Any]` does not match asserted type `Series[str | bytes | int | ... omitted 12 union elements]`
- tests/frame/test_groupby.py:625:15: error[type-assertion-failure] Type `Series[Any]` does not match asserted type `Series[str | bytes | int | ... omitted 12 union elements]`
- Found 4353 diagnostics
+ Found 4351 diagnostics

No memory usage changes detected ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 15, 2026

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-return-type 3 1 4
no-matching-overload 0 7 0
unresolved-attribute 0 3 0
possibly-missing-attribute 0 1 0
Total 3 12 4

Full report with detailed diff (timing results)

@AlexWaygood AlexWaygood force-pushed the alex/rhs-type-narrow branch 2 times, most recently from 1ff64d0 to 0ec7b35 Compare January 16, 2026 12:58
@AlexWaygood AlexWaygood marked this pull request as ready for review January 16, 2026 13:07
Copy link
Member

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice (and great summary).

@AlexWaygood AlexWaygood enabled auto-merge (squash) January 17, 2026 15:46
@AlexWaygood AlexWaygood merged commit 3608c62 into main Jan 17, 2026
48 checks passed
@AlexWaygood AlexWaygood deleted the alex/rhs-type-narrow branch January 17, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem-analyzer ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants