[red-knot] Understand `typing.Callable` by dhruvmanila · Pull Request #16493 · astral-sh/ruff

dhruvmanila · 2025-03-04T09:16:33Z

Summary

This PR implements a general callable type that wraps around a Signature and it uses that new type to represent typing.Callable.

It also implements Display support for Callable. The format is as:

([<arg name>][: <arg type>][ = <default type>], ...) -> <return type>

The / and * separators are added at the correct boundary for positional-only and keyword-only parameters. Now, as typing.Callable only has positional-only parameters, the rendered signature would be:

Callable[[int, str], None]
# (int, str, /) -> None

The / separator represents that all the arguments are positional-only.

The relationship methods that check assignability, subtype relationship, etc. are not yet implemented and will be done so as a follow-up.

Test Plan

Add test cases for display support for Signature and various mdtest for typing.Callable.

github-actions · 2025-03-05T10:44:19Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

crates/red_knot_python_semantic/src/types/infer.rs

carljm

Very cool! Great work.

crates/red_knot_python_semantic/resources/mdtest/annotations/callable.md

crates/red_knot_python_semantic/src/types/infer.rs

crates/red_knot_python_semantic/src/types/signatures.rs

crates/red_knot_python_semantic/src/types/type_ordering.rs

crates/red_knot_python_semantic/src/types.rs

crates/red_knot_python_semantic/src/types/display.rs

dhruvmanila · 2025-03-06T12:04:14Z

Post review changes

Use (...) -> Unknown consistently
Display ... for gradual form. This is done using an additional flag on Parameters to indicate the gradual form.
Use Any for gradual form while Unknown for invalid forms
Avoid special casing for param spec variable
Don't error on bare Callable, infer it as (...) -> Unknown
Remove various todos, add a couple more
Improve display handling of Signature

~~Regarding the relation implementation, I think I'd prefer to add it in a follow-up PR to keep the changes separate and easier to review.~~ I implemented is_single_valued and is_singleton but not member and static_member yet.

carljm

Looks good!

crates/red_knot_python_semantic/resources/mdtest/annotations/callable.md

crates/red_knot_python_semantic/resources/mdtest/invalid_syntax.md

crates/red_knot_python_semantic/src/types.rs

crates/red_knot_python_semantic/src/types/infer.rs

crates/red_knot_python_semantic/src/types/signatures.rs

crates/red_knot_python_semantic/resources/mdtest/annotations/callable.md

* main: [red-knot] Understand `typing.Callable` (#16493) [red-knot] Support unpacking `with` target (#16469) [red-knot] Attribute access and the descriptor protocol (#16416) [`pep8-naming`] Add links to `ignore-names` options in various rules' documentation (#16557) [red-knot] avoid inferring types if unpacking fails (#16530)

AlexWaygood

two tiny nits I spotted that probably don't warrant their own PR, but could maybe be addressed if you're doing further followup work on Callable types

crates/red_knot_python_semantic/src/types/display.rs

AlexWaygood · 2025-03-10T13:49:28Z

crates/red_knot_python_semantic/src/types/signatures.rs

+    pub(crate) fn is_keyword_only(&self) -> bool {
+        matches!(self.kind, ParameterKind::KeywordOnly { .. })
+    }
+
+    pub(crate) fn is_positional_only(&self) -> bool {


nit

Suggested change

pub(crate) fn is_keyword_only(&self) -> bool {

matches!(self.kind, ParameterKind::KeywordOnly { .. })

}

pub(crate) fn is_positional_only(&self) -> bool {

pub(crate) const fn is_keyword_only(&self) -> bool {

matches!(self.kind, ParameterKind::KeywordOnly { .. })

}

pub(crate) const fn is_positional_only(&self) -> bool {

I might be wrong but I'm not sure if this is going to make any difference as we don't use any of these functions in the const context and I don't know we'll ever be able to (?) because parameters are always going to be dynamic i.e., we cannot statically determine during compile time whether a parameter is positional or not.

## Summary This PR is a follow-up to #16493 that implements member lookup for the general callable type. Based on the discussion around [member lookup here](#16493 (comment)) and [`.to_meta_type()` here](#16493 (comment)). ## Test Plan Add a new test cases.

## Summary Currently this assertion fails on `main`, because we do not synthesize a `__call__` attribute for Callable types: ```py from typing import Protocol, Callable from knot_extensions import static_assert, is_assignable_to class Foo(Protocol): def __call__(self, x: int, /) -> str: ... static_assert(is_assignable_to(Callable[[int], str], Foo)) ``` This PR fixes that. See previous discussion about this in #16493 (comment) and #17682 (comment) ## Test Plan Existing mdtests updated; a couple of new ones added.

dhruvmanila added the ty Multi-file analysis & type inference label Mar 4, 2025

dhruvmanila force-pushed the dhruv/callable-type branch from e71d87b to dbae500 Compare March 5, 2025 08:44

[red-knot] Understand typing.Callable

05a7a5b

dhruvmanila force-pushed the dhruv/callable-type branch from dbae500 to 05a7a5b Compare March 5, 2025 10:35

dhruvmanila added 4 commits March 5, 2025 16:17

fix pre-commit

12c8d2e

Move invalid syntax test case

dec9d49

Avoid todo!() macro which panics

ff7d3ce

remove unused method

0ae7e87

dhruvmanila commented Mar 5, 2025

View reviewed changes

crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved

crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved

crates/red_knot_python_semantic/src/types/infer.rs Show resolved Hide resolved

dhruvmanila marked this pull request as ready for review March 5, 2025 17:37

dhruvmanila requested review from AlexWaygood, MichaReiser, carljm and sharkdp as code owners March 5, 2025 17:38

MichaReiser mentioned this pull request Mar 5, 2025

[red-knot] Avoid running ecosystem checks when not necessary #16522

Closed

carljm reviewed Mar 5, 2025

View reviewed changes

MichaReiser approved these changes Mar 6, 2025

View reviewed changes

dhruvmanila added 6 commits March 6, 2025 14:26

Address review feedback

89a33fb

Impl union element ordering

2b9ee87

continue addressing feedback

1d9a6b1

Avoid invalid type form error for literals

44aec36

fix pre-commit

a23220f

update docs

926fa8e

Use (...) -> Unknown consistently

e8c9327

dhruvmanila requested a review from carljm March 6, 2025 15:57

dhruvmanila added 2 commits March 7, 2025 10:16

simplify error message

50cb8c8

Implement is_singleton, is_single_valued

871bddb

carljm approved these changes Mar 7, 2025

View reviewed changes

dhruvmanila mentioned this pull request Mar 7, 2025

[red-knot] Support callable types and typing.Callable #15382

Closed

19 tasks

dhruvmanila added 2 commits March 8, 2025 09:18

Address review feedback

5be89b1

Merge branch 'main' into dhruv/callable-type

94feae3

dhruvmanila enabled auto-merge (squash) March 8, 2025 03:53

fix clippy

47a4a4a

dhruvmanila merged commit 0361021 into main Mar 8, 2025
20 checks passed

dhruvmanila deleted the dhruv/callable-type branch March 8, 2025 03:58

dhruvmanila linked an issue Mar 8, 2025 that may be closed by this pull request

[red-knot] Support callable types and typing.Callable #15382

Closed

19 tasks

AlexWaygood reviewed Mar 10, 2025

View reviewed changes

dhruvmanila mentioned this pull request Mar 11, 2025

[red-knot] Callable member lookup, meta type impl #16618

Merged

AlexWaygood mentioned this pull request May 3, 2025

[red-knot] Synthesize a __call__ attribute for Callable types #17809

Merged

Comments

Conversation

dhruvmanila commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

github-actions bot commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ruff-ecosystem results

Linter (stable)

Linter (preview)

Formatter (stable)

Formatter (preview)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carljm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dhruvmanila commented Mar 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Post review changes

Uh oh!

carljm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AlexWaygood Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

dhruvmanila Mar 11, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dhruvmanila commented Mar 4, 2025 •

edited

Loading

github-actions bot commented Mar 5, 2025 •

edited

Loading

`ruff-ecosystem` results

dhruvmanila commented Mar 6, 2025 •

edited

Loading