[ty] Conservative narrowing places optimization by ericmarkmartin · Pull Request #22734 · astral-sh/ruff

ericmarkmartin · 2026-01-19T19:51:57Z

Summary

Add helpers to compute a conservative upper bound on the set of places a predicate might actually narrow, and use this to save work when computing narrowing constraints.

Test Plan

Existing tests pass

astral-sh-bot · 2026-01-19T20:48:39Z

Typing conformance results

No changes detected ✅

astral-sh-bot · 2026-01-19T20:49:42Z

`mypy_primer` results

Changes were detected when running on open source projects

prefect (https://github.com/PrefectHQ/prefect)
+ src/integrations/prefect-docker/tests/test_containers.py:27:47: error[invalid-argument-type] Argument is incorrect: Expected `str`, found `str | bool | dict[Unknown | str, Unknown | int] | None`
+ src/integrations/prefect-docker/tests/test_containers.py:27:47: error[invalid-argument-type] Argument is incorrect: Expected `str | list[str] | None`, found `str | bool | dict[Unknown | str, Unknown | int] | None`
+ src/integrations/prefect-docker/tests/test_containers.py:27:47: error[invalid-argument-type] Argument is incorrect: Expected `str | None`, found `str | bool | dict[Unknown | str, Unknown | int] | None`
+ src/integrations/prefect-docker/tests/test_containers.py:27:47: error[invalid-argument-type] Argument is incorrect: Expected `bool | None`, found `str | bool | dict[Unknown | str, Unknown | int] | None`
+ src/integrations/prefect-docker/tests/test_containers.py:27:47: error[invalid-argument-type] Argument is incorrect: Expected `str | list[str] | None`, found `str | bool | dict[Unknown | str, Unknown | int] | None`
+ src/integrations/prefect-docker/tests/test_containers.py:27:47: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, str] | list[str] | None`, found `str | bool | dict[Unknown | str, Unknown | int] | None`
+ src/integrations/prefect-docker/tests/test_containers.py:27:47: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `str | bool | dict[Unknown | str, Unknown | int] | None`
+ src/integrations/prefect-docker/tests/test_containers.py:42:47: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `str`
+ src/integrations/prefect-docker/tests/test_containers.py:55:47: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `str`
+ src/integrations/prefect-docker/tests/test_containers.py:68:47: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `str`
+ src/integrations/prefect-docker/tests/test_containers.py:81:47: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `str`
+ src/integrations/prefect-docker/tests/test_images.py:16:44: error[invalid-argument-type] Argument is incorrect: Expected `str`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:16:44: error[invalid-argument-type] Argument is incorrect: Expected `str | None`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:16:44: error[invalid-argument-type] Argument is incorrect: Expected `str | None`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:16:44: error[invalid-argument-type] Argument is incorrect: Expected `bool`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:16:44: error[invalid-argument-type] Argument is incorrect: Expected `DockerHost | None`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:16:44: error[invalid-argument-type] Argument is incorrect: Expected `DockerRegistryCredentials | None`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:16:44: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:21:16: warning[possibly-missing-attribute] Attribute `id` may be missing on object of type `Unknown | list[Unknown]`
+ src/integrations/prefect-docker/tests/test_images.py:29:47: error[invalid-argument-type] Argument is incorrect: Expected `bool`, found `str`
+ src/integrations/prefect-docker/tests/test_images.py:29:47: error[invalid-argument-type] Argument is incorrect: Expected `DockerRegistryCredentials | None`, found `str`
+ src/integrations/prefect-docker/tests/test_images.py:29:47: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `str`
+ src/integrations/prefect-docker/tests/test_images.py:31:16: warning[possibly-missing-attribute] Attribute `id` may be missing on object of type `Unknown | list[Unknown]`
+ src/integrations/prefect-docker/tests/test_images.py:51:17: error[invalid-argument-type] Argument is incorrect: Expected `bool`, found `str`
+ src/integrations/prefect-docker/tests/test_images.py:51:17: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `str`
+ src/integrations/prefect-docker/tests/test_images.py:53:16: warning[possibly-missing-attribute] Attribute `id` may be missing on object of type `Unknown | list[Unknown]`
+ src/integrations/prefect-docker/tests/test_images.py:64:47: error[invalid-argument-type] Argument is incorrect: Expected `str`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:64:47: error[invalid-argument-type] Argument is incorrect: Expected `str | None`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:64:47: error[invalid-argument-type] Argument is incorrect: Expected `str | None`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:64:47: error[invalid-argument-type] Argument is incorrect: Expected `bool`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:64:47: error[invalid-argument-type] Argument is incorrect: Expected `DockerRegistryCredentials | None`, found `str | bool`
+ src/integrations/prefect-docker/tests/test_images.py:64:47: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `str | bool`
+ src/integrations/prefect-kubernetes/prefect_kubernetes/jobs.py:428:17: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `str`
+ src/integrations/prefect-kubernetes/tests/test_custom_objects.py:20:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `None`
+ src/integrations/prefect-kubernetes/tests/test_custom_objects.py:29:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `None`
+ src/integrations/prefect-kubernetes/tests/test_custom_objects.py:38:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `None`
+ src/integrations/prefect-kubernetes/tests/test_custom_objects.py:57:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_custom_objects.py:103:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_custom_objects.py:149:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_custom_objects.py:195:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_custom_objects.py:240:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_custom_objects.py:286:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_custom_objects.py:344:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_deployments.py:18:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_deployments.py:38:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_deployments.py:70:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_deployments.py:92:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_deployments.py:113:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_deployments.py:141:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_jobs.py:36:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_jobs.py:52:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_jobs.py:68:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_jobs.py:87:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_jobs.py:107:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_jobs.py:131:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_jobs.py:159:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_pods.py:29:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_pods.py:46:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_pods.py:78:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_pods.py:96:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_pods.py:115:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_pods.py:137:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
+ src/integrations/prefect-kubernetes/tests/test_pods.py:167:9: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Any]`, found `Literal["test"]`
- src/prefect/cache_policies.py:311:25: warning[possibly-missing-attribute] Attribute `__code__` may be missing on object of type `Unknown | (((...) -> Any) & ((*args: object, **kwargs: object) -> object))`
+ src/prefect/cache_policies.py:311:25: warning[possibly-missing-attribute] Attribute `__code__` may be missing on object of type `Unknown | ((...) -> Any)`
- src/prefect/input/run_input.py:672:20: error[invalid-return-type] Return type does not match returned value: expected `T@GetAutomaticInputHandler | AutomaticRunInput[T@GetAutomaticInputHandler]`, found `T@GetAutomaticInputHandler | AutomaticRunInput[T@GetAutomaticInputHandler] | Coroutine[Any, Any, T@GetAutomaticInputHandler | AutomaticRunInput[T@GetAutomaticInputHandler]]`
+ src/prefect/input/run_input.py:672:20: error[invalid-return-type] Return type does not match returned value: expected `T@GetAutomaticInputHandler | AutomaticRunInput[T@GetAutomaticInputHandler]`, found `Unknown | Coroutine[Any, Any, Unknown]`
+ src/prefect/task_engine.py:1641:28: error[invalid-await] `Unknown | R@AsyncTaskRunEngine | Coroutine[Any, Any, R@AsyncTaskRunEngine]` is not awaitable
+ src/prefect/task_engine.py:1749:47: error[invalid-argument-type] Argument to function `next` is incorrect: Expected `SupportsNext[Unknown]`, found `Unknown | R@run_generator_task_sync`
+ src/prefect/task_engine.py:1762:25: warning[possibly-missing-attribute] Attribute `throw` may be missing on object of type `Unknown | R@run_generator_task_sync`
+ src/prefect/task_engine.py:1808:48: warning[possibly-missing-attribute] Attribute `__anext__` may be missing on object of type `Unknown | R@run_generator_task_async`
+ src/prefect/task_engine.py:1820:29: warning[possibly-missing-attribute] Attribute `throw` may be missing on object of type `Unknown | R@run_generator_task_async`
- src/prefect/tasks.py:185:9: warning[possibly-missing-attribute] Attribute `__code__` may be missing on object of type `Unknown | (((...) -> Any) & ((*args: object, **kwargs: object) -> object))`
+ src/prefect/tasks.py:185:9: warning[possibly-missing-attribute] Attribute `__code__` may be missing on object of type `Unknown | ((...) -> Any)`
- Found 5393 diagnostics
+ Found 5461 diagnostics

scikit-build-core (https://github.com/scikit-build/scikit-build-core)
+ src/scikit_build_core/build/wheel.py:99:20: error[no-matching-overload] No overload of bound method `__init__` matches arguments
- Found 49 diagnostics
+ Found 50 diagnostics

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
- tests/test_pandas.py:511:18: error[assert-type-unspellable-subtype] Type `bool` does not match asserted type `TypeIs[timedelta64[timedelta & ~NaTType & ~NAType] @ np_td]`
+ tests/test_pandas.py:511:18: error[assert-type-unspellable-subtype] Type `bool` does not match asserted type `TypeIs[timedelta64[timedelta] @ np_td]`
- tests/test_pandas.py:516:22: error[assert-type-unspellable-subtype] Type `bool` does not match asserted type `TypeIs[str | bytes | date | ... omitted 9 union elements @ np_nat]`
+ tests/test_pandas.py:516:22: error[assert-type-unspellable-subtype] Type `bool` does not match asserted type `TypeIs[timedelta64[timedelta | int | None] @ np_nat]`

Memory usage changes were detected when running on open source projects

flake8 (https://github.com/pycqa/flake8)
- TOTAL MEMORY USAGE: ~66MB
+ TOTAL MEMORY USAGE: ~63MB

trio (https://github.com/python-trio/trio)
- TOTAL MEMORY USAGE: ~167MB
+ TOTAL MEMORY USAGE: ~159MB
-     struct metadata = ~11MB
+     struct metadata = ~10MB
-     memo fields = ~108MB
+     memo fields = ~103MB

sphinx (https://github.com/sphinx-doc/sphinx)
- TOTAL MEMORY USAGE: ~301MB
+ TOTAL MEMORY USAGE: ~287MB
-     memo fields = ~176MB
+     memo fields = ~167MB

prefect (https://github.com/PrefectHQ/prefect)
-     memo fields = ~424MB
+     memo fields = ~403MB

AlexWaygood · 2026-01-26T22:31:42Z

Codspeed's reporting speedups of up to 4%, and this also looks like a pretty big win in terms of memory usage according to the primer report!

ericmarkmartin · 2026-01-27T01:15:27Z

looks like a pretty big win in terms of memory usage according to the primer report!

Oh huh that's neat! The only cautionary note I have about this PR is that there's no particularly good mechanism in place for keeping the conservative places in line with the actually inferred places. Of course if we started narrowing a place that's not in the conservative place set then ty won't actually produce the narrowing, so that should be relatively catchable at test-time

carljm · 2026-01-30T15:08:13Z

@ericmarkmartin is anything lacking to put this out of draft and up for review?

ericmarkmartin · 2026-02-05T04:40:06Z

@ericmarkmartin is anything lacking to put this out of draft and up for review?

No, not really, just still wasn't sure if we wanted this without the fix for #690

sharkdp

This is a really good optimization idea and a great implementation — thank you very much!

crates/ty_python_semantic/src/semantic_index/use_def.rs

crates/ty_python_semantic/src/types/narrow.rs

codspeed-hq · 2026-02-05T14:35:55Z

CodSpeed Performance Report

Merging this PR will improve performance by 4.18%

_{Comparing ericmarkmartin:synthetic-narrow-definitions (467aef4) with main (789399c)}

Summary

⚡ 1 improved benchmark
✅ 23 untouched benchmarks
⏩ 30 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	Simulation	`DateType`	241.5 ms	231.8 ms	+4.18%

30 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

AlexWaygood added the ty Multi-file analysis & type inference label Jan 19, 2026

ericmarkmartin force-pushed the synthetic-narrow-definitions branch from e7da68d to 2136d82 Compare January 19, 2026 20:40

ericmarkmartin added 3 commits January 19, 2026 15:45

conservative narrowing places

dd877aa

narrow only to conservative places

35d4ab5

simplify

143779d

ericmarkmartin force-pushed the synthetic-narrow-definitions branch from 2136d82 to 143779d Compare January 19, 2026 20:46

clippy

7a96650

ericmarkmartin changed the title ~~Conservative narrowing places optimization~~ [ty] Conservative narrowing places optimization Jan 19, 2026

AlexWaygood added the performance Potential performance improvement label Jan 26, 2026

ericmarkmartin marked this pull request as ready for review February 5, 2026 04:26

ericmarkmartin requested review from AlexWaygood, carljm, dcreager and sharkdp as code owners February 5, 2026 04:26

sharkdp approved these changes Feb 5, 2026

View reviewed changes

crates/ty_python_semantic/src/semantic_index/use_def.rs Outdated Show resolved Hide resolved

crates/ty_python_semantic/src/types/narrow.rs Outdated Show resolved Hide resolved

crates/ty_python_semantic/src/types/narrow.rs Outdated Show resolved Hide resolved

Minor improvements in the comments

467aef4

sharkdp merged commit c8703cb into astral-sh:main Feb 5, 2026
49 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ty] Conservative narrowing places optimization#22734

[ty] Conservative narrowing places optimization#22734
sharkdp merged 5 commits intoastral-sh:mainfrom
ericmarkmartin:synthetic-narrow-definitions

ericmarkmartin commented Jan 19, 2026 •

edited

Loading

Uh oh!

astral-sh-bot bot commented Jan 19, 2026 •

edited

Loading

Uh oh!

astral-sh-bot bot commented Jan 19, 2026 •

edited

Loading

Uh oh!

AlexWaygood commented Jan 26, 2026 •

edited

Loading

Uh oh!

ericmarkmartin commented Jan 27, 2026 •

edited

Loading

Uh oh!

carljm commented Jan 30, 2026

Uh oh!

ericmarkmartin commented Feb 5, 2026

Uh oh!

sharkdp left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codspeed-hq bot commented Feb 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

ericmarkmartin commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

astral-sh-bot bot commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Typing conformance results

Uh oh!

astral-sh-bot bot commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mypy_primer results

Uh oh!

AlexWaygood commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ericmarkmartin commented Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

carljm commented Jan 30, 2026

Uh oh!

ericmarkmartin commented Feb 5, 2026

Uh oh!

sharkdp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codspeed-hq bot commented Feb 5, 2026

CodSpeed Performance Report

Merging this PR will improve performance by 4.18%

Summary

Performance Changes

Footnotes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ericmarkmartin commented Jan 19, 2026 •

edited

Loading

astral-sh-bot bot commented Jan 19, 2026 •

edited

Loading

astral-sh-bot bot commented Jan 19, 2026 •

edited

Loading

`mypy_primer` results

AlexWaygood commented Jan 26, 2026 •

edited

Loading

ericmarkmartin commented Jan 27, 2026 •

edited

Loading