[ty] Improve `UnionBuilder` performance by changing `Type::is_subtype_of` calls to `Type::is_redundant_with` by AlexWaygood · Pull Request #22337 · astral-sh/ruff

AlexWaygood · 2026-01-02T16:19:20Z

Summary

This PR is stacked on top of #22339; review that one first.

Type::is_redundant_with has mostly the same semantics as Type::is_subtype_of, but:

More aggressively simplifies unions in some situations
Is cached!

There are several places in the UnionBuilder where it looks like we can safely used Type::is_redundant_with instead of Type::is_subtype_of. This significantly improves performance, has no impact on our test suite, and (according to mypy_primer) has no impact on any ecosystem diagnostics or ty's memory usage. (The reported changes in diagnostics are all just our standard ecosystem flakes.)

astral-sh-bot · 2026-01-02T16:21:36Z

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

astral-sh-bot · 2026-01-02T16:22:46Z

`mypy_primer` results

Changes were detected when running on open source projects

prefect (https://github.com/PrefectHQ/prefect)
- src/prefect/deployments/runner.py:795:70: warning[possibly-missing-attribute] Attribute `__name__` may be missing on object of type `Unknown | (((...) -> Any) & ((*args: object, **kwargs: object) -> object))`
+ src/prefect/deployments/runner.py:795:70: warning[possibly-missing-attribute] Attribute `__name__` may be missing on object of type `Unknown | ((...) -> Any)`
+ src/prefect/flow_engine.py:812:32: error[invalid-await] `Unknown | R@FlowRunEngine | Coroutine[Any, Any, R@FlowRunEngine]` is not awaitable
+ src/prefect/flow_engine.py:1401:24: error[invalid-await] `Unknown | R@AsyncFlowRunEngine | Coroutine[Any, Any, R@AsyncFlowRunEngine]` is not awaitable
+ src/prefect/flow_engine.py:1482:43: error[invalid-argument-type] Argument to function `next` is incorrect: Expected `SupportsNext[Unknown]`, found `Unknown | R@run_generator_flow_sync`
+ src/prefect/flow_engine.py:1490:21: warning[possibly-missing-attribute] Attribute `throw` may be missing on object of type `Unknown | R@run_generator_flow_sync`
+ src/prefect/flow_engine.py:1524:44: warning[possibly-missing-attribute] Attribute `__anext__` may be missing on object of type `Unknown | R@run_generator_flow_async`
+ src/prefect/flow_engine.py:1531:25: warning[possibly-missing-attribute] Attribute `throw` may be missing on object of type `Unknown | R@run_generator_flow_async`
- src/prefect/flows.py:286:34: error[unresolved-attribute] Object of type `((**P@Flow) -> R@Flow) & ((*args: object, **kwargs: object) -> object)` has no attribute `__name__`
+ src/prefect/flows.py:286:34: error[unresolved-attribute] Object of type `(**P@Flow) -> R@Flow` has no attribute `__name__`
- src/prefect/flows.py:404:68: error[unresolved-attribute] Object of type `((**P@Flow) -> R@Flow) & ((*args: object, **kwargs: object) -> object)` has no attribute `__name__`
+ src/prefect/flows.py:404:68: error[unresolved-attribute] Object of type `(**P@Flow) -> R@Flow` has no attribute `__name__`
- src/prefect/flows.py:1750:53: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- Found 5377 diagnostics
+ Found 5382 diagnostics

static-frame (https://github.com/static-frame/static-frame)
- static_frame/core/bus.py:671:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[Bus[Any], object_]`, found `InterGetItemLocReduces[Bus[Any] | TypeBlocks | Batch | ... omitted 6 union elements, object_]`
- static_frame/core/bus.py:675:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Bus[Any], object_]`, found `InterGetItemILocReduces[Bus[Any] | Bottom[Index[Any]] | TypeBlocks | ... omitted 6 union elements, object_ | Self@iloc]`
+ static_frame/core/bus.py:675:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Bus[Any], object_]`, found `InterGetItemILocReduces[Self@iloc | Bus[Any], object_ | Self@iloc]`
- static_frame/core/index.py:580:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[TVContainer_co@loc, TVDtype@Index]`, found `InterGetItemLocReduces[Any | Bottom[Series[Any, Any]], TVDtype@Index]`
+ static_frame/core/index.py:580:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[TVContainer_co@loc, TVDtype@Index]`, found `InterGetItemLocReduces[Bottom[Series[Any, Any]] | Any, TVDtype@Index]`
- static_frame/core/node_selector.py:526:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[TVContainer_co@InterfaceSelectQuartet, Any]`, found `InterGetItemLocReduces[Unknown | Bottom[Series[Any, Any]], Any]`
+ static_frame/core/node_selector.py:526:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[TVContainer_co@InterfaceSelectQuartet, Any]`, found `InterGetItemLocReduces[Bottom[Series[Any, Any]] | Unknown, Any]`
- static_frame/core/series.py:772:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Series[Any, Any], TVDtype@Series]`, found `InterGetItemILocReduces[Series[Any, Any] | TypeBlocks | Batch | ... omitted 6 union elements, TVDtype@Series]`
+ static_frame/core/series.py:772:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Series[Any, Any], TVDtype@Series]`, found `InterGetItemILocReduces[Series[Any, Any] | Bottom[Index[Any]] | TypeBlocks | ... omitted 6 union elements, TVDtype@Series]`
- static_frame/core/yarn.py:418:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Yarn[Any], object_]`, found `InterGetItemILocReduces[Yarn[Any] | Bottom[Index[Any]] | TypeBlocks | ... omitted 6 union elements, object_]`
- Found 1838 diagnostics
+ Found 1836 diagnostics

core (https://github.com/home-assistant/core)
- homeassistant/util/variance.py:47:12: error[invalid-return-type] Return type does not match returned value: expected `(**_P@ignore_variance) -> _R@ignore_variance`, found `_Wrapped[_P@ignore_variance, _R@ignore_variance | int | float | datetime, _P@ignore_variance, _R@ignore_variance | int | float | datetime]`
- Found 14484 diagnostics
+ Found 14483 diagnostics

Memory usage changes were detected when running on open source projects

trio (https://github.com/python-trio/trio)
-     struct fields = ~11MB
+     struct fields = ~12MB

codspeed-hq · 2026-01-02T16:35:16Z

Merging this PR will improve performance by 17.32%

Summary

⚡ 2 improved benchmarks
✅ 21 untouched benchmarks
⏩ 30 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	WallTime	`altair`	4.9 s	4.6 s	+8.23%
⚡	WallTime	`multithreaded`	1.3 s	1.1 s	+17.32%

_{Comparing alex/more-redundancy (f3b9422) with main (c02d164)}

30 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

MichaReiser

I'd prefer if @carljm reviews this PR too because I'm not very familiar with the semantic differences of the two methods but this now matches the pattern we use in the more general push_type

carljm

Looks good!

It seems a bit subtle when we can use is_redundant_with and when we need is_subtype_of -- there are still several is_subtype_of checks here, one with enums and several when we are checking subsumption by a negated type. I wonder how we could improve this... but I don't think it needs to happen in this PR.

Pushing one doc-comment update.

AlexWaygood · 2026-01-07T22:16:38Z

Looks good!

It seems a bit subtle when we can use is_redundant_with and when we need is_subtype_of -- there are still several is_subtype_of checks here, one with enums and several when we are checking subsumption by a negated type. I wonder how we could improve this... but I don't think it needs to happen in this PR.

Pushing one doc-comment update.

I don't think our simplifications for intersections with negated gradual elements is quite right on main, and I suspect this is why switching to is_redundant_with for the negated-type-subsumption check causes some subtle behaviour changes. I have a very stale PR that attempts to fix some of these issues that I need to get back to at some point (the first version of that PR was incorrect, as David pointed out in review... #20651)

AlexWaygood added performance Potential performance improvement ty Multi-file analysis & type inference labels Jan 2, 2026

AlexWaygood force-pushed the alex/more-redundancy branch 3 times, most recently from fb77ea7 to 0567017 Compare January 2, 2026 16:34

AlexWaygood force-pushed the alex/more-redundancy branch from 0567017 to d1d8fd4 Compare January 2, 2026 16:45

AlexWaygood changed the base branch from main to alex/optimize-try-reduce January 2, 2026 17:06

AlexWaygood changed the title ~~[ty] Improve UnionBuilder performance~~ [ty] Improve UnionBuilder performance by changing Type::is_subtype_of calls to Type::is_redundant_with Jan 2, 2026

AlexWaygood marked this pull request as ready for review January 2, 2026 17:14

AlexWaygood requested review from carljm, dcreager and sharkdp as code owners January 2, 2026 17:14

AlexWaygood force-pushed the alex/optimize-try-reduce branch from c5b12a3 to 6e97c3b Compare January 2, 2026 21:04

AlexWaygood force-pushed the alex/more-redundancy branch from d1d8fd4 to d5812ee Compare January 2, 2026 21:48

MichaReiser approved these changes Jan 3, 2026

View reviewed changes

AlexWaygood force-pushed the alex/more-redundancy branch from d5812ee to 4952a0d Compare January 4, 2026 21:48

Base automatically changed from alex/optimize-try-reduce to main January 5, 2026 12:54

[ty] Improve UnionBuilder performance

d9f7d7a

AlexWaygood force-pushed the alex/more-redundancy branch from 4952a0d to d9f7d7a Compare January 5, 2026 13:15

AlexWaygood and others added 2 commits January 5, 2026 15:16

revert one change...

0df22e7

update doc comment

f3b9422

carljm approved these changes Jan 7, 2026

View reviewed changes

AlexWaygood merged commit abaa735 into main Jan 7, 2026
48 checks passed

AlexWaygood deleted the alex/more-redundancy branch January 7, 2026 22:17

AlexWaygood mentioned this pull request Jan 16, 2026

[ty] Cache some union builder queries #22052

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ty] Improve `UnionBuilder` performance by changing `Type::is_subtype_of` calls to `Type::is_redundant_with`#22337

[ty] Improve `UnionBuilder` performance by changing `Type::is_subtype_of` calls to `Type::is_redundant_with`#22337
AlexWaygood merged 3 commits intomainfrom
alex/more-redundancy

AlexWaygood commented Jan 2, 2026 •

edited

Loading

Uh oh!

astral-sh-bot bot commented Jan 2, 2026 •

edited

Loading

Uh oh!

astral-sh-bot bot commented Jan 2, 2026 •

edited

Loading

Uh oh!

codspeed-hq bot commented Jan 2, 2026 •

edited

Loading

Uh oh!

MichaReiser left a comment

Uh oh!

carljm left a comment

Uh oh!

AlexWaygood commented Jan 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

AlexWaygood commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

astral-sh-bot bot commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Diagnostic diff on typing conformance tests

Uh oh!

astral-sh-bot bot commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mypy_primer results

Uh oh!

codspeed-hq bot commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by 17.32%

Summary

Performance Changes

Footnotes

Uh oh!

MichaReiser left a comment

Choose a reason for hiding this comment

Uh oh!

carljm left a comment

Choose a reason for hiding this comment

Uh oh!

AlexWaygood commented Jan 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AlexWaygood commented Jan 2, 2026 •

edited

Loading

astral-sh-bot bot commented Jan 2, 2026 •

edited

Loading

astral-sh-bot bot commented Jan 2, 2026 •

edited

Loading

`mypy_primer` results

codspeed-hq bot commented Jan 2, 2026 •

edited

Loading