Now `TypeInfo.get_method` also returns `Decorator` nodes by sobolevn · Pull Request #11150 · python/mypy

sobolevn · 2021-09-20T15:45:46Z

Closes #10409

JukkaL

Thanks for the PR! Left a few comments and a question.

test-data/unit/check-classes.test

mypy/checkmember.py

sobolevn · 2021-09-21T17:53:08Z

This commit just addresses the code review: 28d950d

Now, I am going to change how .get_method() is defined. Let's see what will happen!

sobolevn · 2021-09-21T18:22:45Z

I am running self-check with the only change: def get_method(self) -> Union[FuncBase, Decorator, None]

» python runtests.py self
run self: ['/Users/sobolev/Desktop/mypy/.venv/bin/python', '-m', 'mypy', '--config-file', 'mypy_self_check.ini', '-p', 'mypy']
mypy/subtypes.py:618: error: Item "Decorator" of "Union[FuncBase, Decorator]" has no
attribute "is_property"  [union-attr]
            if method.is_property:
               ^
mypy/subtypes.py:623: error: Argument 1 to "find_node_type" has incompatible type
"Union[FuncBase, Decorator]"; expected "Union[Var, FuncBase]"  [arg-type]
            return find_node_type(method, itype, subtype)
                                  ^
mypy/subtypes.py:644: error: Argument 1 to "find_node_type" has incompatible type
"Union[FuncBase, Decorator]"; expected "Union[Var, FuncBase]"  [arg-type]
    ...         getattr_type = get_proper_type(find_node_type(method, itype, subtyp...
                                                              ^
mypy/subtypes.py:667: error: Item "Decorator" of "Union[FuncBase, Decorator]" has no
attribute "is_property"  [union-attr]
            if method.is_property:
               ^
mypy/checkmember.py:199: error: Item "Decorator" of "Union[FuncBase, Decorator]" has
no attribute "is_property"  [union-attr]
            if method.is_property:
               ^
mypy/checkmember.py:205: error: Argument 1 to "function_type" has incompatible type
"Union[FuncBase, Decorator]"; expected "FuncBase"  [arg-type]
            signature = function_type(method, mx.builtin_type('builtins.function...
                                      ^
mypy/checkmember.py:216: error: Item "Decorator" of "Union[FuncBase, Decorator]" has
no attribute "is_class"  [union-attr]
    ...     signature = check_self_arg(signature, dispatched_type, method.is_class,
                                                                   ^
mypy/checkmember.py:218: error: Item "Decorator" of "Union[FuncBase, Decorator]" has
no attribute "is_class"  [union-attr]
    ...gnature = bind_self(signature, mx.self_type, is_classmethod=method.is_class)
                                                                   ^
mypy/checkmember.py:392: error: Argument 1 to "function_type" has incompatible type
"Union[FuncBase, Decorator]"; expected "FuncBase"  [arg-type]
                        function = function_type(method, mx.builtin_type('builti...
                                                 ^
mypy/checkmember.py:409: error: Argument 1 to "function_type" has incompatible type
"Union[FuncBase, Decorator]"; expected "FuncBase"  [arg-type]
                    setattr_func = function_type(setattr_meth, mx.builtin_type('...
                                                 ^
mypy/checkmember.py:479: error: Argument 1 to "function_type" has incompatible type
"Union[FuncBase, Decorator]"; expected "FuncBase"  [arg-type]
        function = function_type(dunder_get, builtin_type('builtins.function'))
                                 ^
mypy/checker.py:3125: error: Argument 1 to "function_type" has incompatible type
"Union[FuncBase, Decorator]"; expected "FuncBase"  [arg-type]
            function = function_type(dunder_set, self.named_type('builtins.funct...
                                     ^
mypy/checker.py:6042: error: Never apply isinstance() to unexpanded types; use
mypy.types.get_proper_type() first  [misc]
                if isinstance(method.type, Overloaded):
                   ^
mypy/checker.py:6042: note: If you pass on the original type after the check, always use its unexpanded version
Found 13 errors in 3 files (checked 164 source files)

FAILED: self

Looks doable! 👍

sobolevn · 2021-09-22T10:16:37Z

This is now pretty big:

I've changed all parts where .get_method() was used to work with decorators, including

Protocol checks with find_member
is_untyped_decorator function in checker
checkmember

I've changed how type was inferenced from decorators, so for example decorating x.__getattr__ actually changes what type is inferred for x.attr
I've added new tests for each individual change, improved some existing ones

sobolevn · 2021-09-22T10:18:03Z

mypy/checkmember.py

+                    else:
+                        result = getattr_type
+
+                    # Call the attribute hook before returning.


Now plugin hooks are also triggered for non callable types as well.

sobolevn · 2021-09-22T10:34:05Z

Ok, looks like mypyc also need to be fixed. It is failing right now https://github.com/python/mypy/runs/3673913236

@JukkaL any tips on that? I haven't used mypyc 😞

sobolevn · 2021-09-22T14:30:46Z

mypyc didn't like this code:

if isinstance(node, (FuncBase, Decorator)):
    return node

But, it accepted this code:

if isinstance(node, FuncBase):
    return node
elif isinstance(node, Decorator):
    return node

Seems like a bug in mypyc to me.

JukkaL · 2021-09-24T16:55:49Z

I think I found a reproducer for the mypyc issue: mypyc/mypyc#892

sobolevn · 2021-09-29T09:01:35Z

@JukkaL awesome! Thanks! Is there anything else left to do here?

sobolevn · 2021-10-02T16:09:12Z

Rebased on master 🚆

sobolevn · 2021-10-13T20:34:52Z

@JukkaL any chance to get this merged? 🙂

sobolevn · 2021-11-06T15:04:47Z

@JukkaL rebased once again 🙂

I will also friendly ping @hauntsaninja as well.

97littleleaf11

Using MemberContext in analyze_descriptor_access is a good refactoring!

JukkaL

Thanks for the PR, this looks solid! I like how this fixes many related issues.

JukkaL · 2021-12-03T13:18:09Z

mypy/checker.py

-
-        function = function_type(dunder_set, self.named_type('builtins.function'))
-        bound_method = bind_self(function, attribute_type)
+        if isinstance(dunder_set, Decorator):


Code quite similar to this if/else gets repeated several times. I wonder if it would be possible to abstract it into a helper function, for example? If this sounds like a feasible idea, it could be a nice follow-up PR.

sobolevn · 2021-12-03T13:24:54Z

@JukkaL thank you! I will think about a helper function 🙂

* Follow up to #11150, refactors `if`s into a util function

Support decorators properly in additional contexts. Closes python#10409

* Follow up to python#11150, refactors `if`s into a util function

JukkaL reviewed Sep 21, 2021

View reviewed changes

test-data/unit/check-classes.test Outdated Show resolved Hide resolved

test-data/unit/check-classes.test Show resolved Hide resolved

test-data/unit/check-classes.test Outdated Show resolved Hide resolved

mypy/checkmember.py Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

sobolevn commented Sep 22, 2021

View reviewed changes

sobolevn changed the title ~~Now decorated __getattr__ works~~ Now TypeInfo.get_method also returns Decorator nodes Sep 22, 2021

sobolevn mentioned this pull request Sep 22, 2021

callable is not documented in type narrowing section #11172

Closed

JukkaL mentioned this pull request Sep 24, 2021

isinstance failure with complex class hierarchy mypyc/mypyc#892

Closed

sobolevn added 8 commits October 2, 2021 19:06

Now decorated __getattr__ works, refs #10409

362b710

Addresses code review

96d9ec8

Experiment: get_method() returns decorated methods

374ef19

Reworks how TypeInfo.get_method() works

750eefc

Improves __get__ and __set__ tests

500ae57

Improves callable testing

474429c

Fixes self type checking

1ea795d

Fixes mypyc

662c342

Typo fixed

d24b9ae

This comment has been minimized.

Sign in to view

sobolevn added 2 commits November 6, 2021 17:54

Merge branch 'master' into issue-10409

7cfb522

Merge into master

44b565a

This comment has been minimized.

Sign in to view

97littleleaf11 self-requested a review November 12, 2021 17:42

97littleleaf11 reviewed Dec 1, 2021

View reviewed changes

JukkaL approved these changes Dec 3, 2021

View reviewed changes

JukkaL merged commit 4e34fec into python:master Dec 3, 2021

sobolevn mentioned this pull request Dec 4, 2021

Follow up to #11150, refactors ifs into util function #11663

Merged

97littleleaf11 pushed a commit that referenced this pull request Dec 5, 2021

Refactor ifs into util function (#11663)

f1eb04a

* Follow up to #11150, refactors `if`s into a util function

tushar-deepsource pushed a commit to DeepSourceCorp/mypy that referenced this pull request Jan 20, 2022

Now TypeInfo.get_method also returns Decorator nodes (python#11150)

3f6a197

Support decorators properly in additional contexts. Closes python#10409

tushar-deepsource pushed a commit to DeepSourceCorp/mypy that referenced this pull request Jan 20, 2022

Refactor ifs into util function (python#11663)

aafe7bd

* Follow up to python#11150, refactors `if`s into a util function

sobolevn mentioned this pull request Jan 20, 2022

Fix crashes in class scoped imports #12023

Merged

med1844 mentioned this pull request May 19, 2025

[ty] Add assignability between class with __call__ class attribute and callable astral-sh/ruff#18167

Closed

Uh oh!

Comments

Conversation

sobolevn commented Sep 20, 2021

Uh oh!

JukkaL left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sobolevn commented Sep 21, 2021

Uh oh!

This comment has been minimized.

sobolevn commented Sep 21, 2021

Uh oh!

sobolevn commented Sep 22, 2021

Uh oh!

sobolevn Sep 22, 2021

Choose a reason for hiding this comment

Uh oh!

sobolevn commented Sep 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sobolevn commented Sep 22, 2021

Uh oh!

JukkaL commented Sep 24, 2021

Uh oh!

sobolevn commented Sep 29, 2021

Uh oh!

sobolevn commented Oct 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

sobolevn commented Oct 13, 2021

Uh oh!

sobolevn commented Nov 6, 2021

Uh oh!

This comment has been minimized.

97littleleaf11 left a comment

Choose a reason for hiding this comment

Uh oh!

JukkaL left a comment

Choose a reason for hiding this comment

Uh oh!

JukkaL Dec 3, 2021

Choose a reason for hiding this comment

Uh oh!

sobolevn commented Dec 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sobolevn commented Sep 22, 2021 •

edited

Loading

sobolevn commented Oct 2, 2021 •

edited

Loading