[red-knot] Method calls and the descriptor protocol#16121
Merged
Conversation
f8bf60f to
00d04b1
Compare
sharkdp
commented
Feb 17, 2025
crates/red_knot_python_semantic/resources/mdtest/annotations/literal_string.md
Show resolved
Hide resolved
CodSpeed Performance ReportMerging #16121 will degrade performances by 6.88%Comparing Summary
Benchmarks breakdown
|
This comment was marked as resolved.
This comment was marked as resolved.
sharkdp
commented
Feb 17, 2025
sharkdp
commented
Feb 17, 2025
sharkdp
commented
Feb 17, 2025
sharkdp
commented
Feb 17, 2025
sharkdp
commented
Feb 17, 2025
crates/red_knot_python_semantic/resources/mdtest/scopes/moduletype_attrs.md
Outdated
Show resolved
Hide resolved
sharkdp
commented
Feb 18, 2025
sharkdp
commented
Feb 18, 2025
crates/red_knot_python_semantic/resources/mdtest/descriptor_protocol.md
Outdated
Show resolved
Hide resolved
sharkdp
commented
Feb 18, 2025
carljm
reviewed
Feb 19, 2025
Contributor
carljm
left a comment
There was a problem hiding this comment.
This is looking pretty good to me! Left a few comments.
crates/red_knot_python_semantic/resources/mdtest/scopes/moduletype_attrs.md
Outdated
Show resolved
Hide resolved
3056b98 to
e3f2ad2
Compare
e3f2ad2 to
e61e653
Compare
MichaReiser
approved these changes
Feb 19, 2025
e61e653 to
d84b1fe
Compare
7e90171 to
92416e1
Compare
41d1362 to
05d689d
Compare
This was referenced Feb 21, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR achieves the following:
__get__method when a descriptor object is accessed through a class object or an instance of a class. For example:inspect.getattr_static(obj, attr)calls. This was mostly used as a debugging tool during development, but seems more generally useful. It can be used to bypass the descriptor protocol. For the example above:Type::Callable(…)variant with the following sub-variants:Type::Callable(CallableType::BoundMethod(…))— represents bound method objects, e.g.C().faboveType::Callable(CallableType::MethodWrapperDunderGet(…))— representsf.__get__wherefis a functionType::Callable(WrapperDescriptorDunderGet)— representsFunctionType.__get__types.MethodTypetypes.MethodWrapperTypetypes.WrapperDescriptorTypebuiltins.rangePerformance analysis
On this branch, we do more work. We need to do more call checking, since we now check all method calls. We also need to do ~twice as many member lookups, because we need to check if a
__get__attribute exists on accessed members.A brief analysis on
tomllibshows that we now callType::call1780 times, compared to 612 calls before.We also emit 18 additional diagnostics, which might also explain some of the performance regression.I did some wall time benchmarks as well and only find a minor 1% regression on
black, and a 6% regression ontomllib.black./red_knot_feature check --project ~/black./red_knot_main check --project ~/blacktomllib./red_knot_feature check --project ~/tomllib./red_knot_main check --project ~/tomllibLimitations
Storecontext and do not check writes to descriptor attributes. I felt like this was something that could be split out as a follow-up without risking a major architectural change.Type::member(with descriptor protocol) andType::static_member(without descriptor protocol). The former corresponds toobj.attr, the latter corresponds togetattr_static(obj, "attr"). However, to model some details correctly, we would also need to distinguish between a static member lookup with and without instance variables. The lookup without instance variables corresponds tofind_name_in_mrohere. We currently approximate both usingmember_static, which leads to two open TODOs. Changing this would be a larger refactoring ofType::own_instance_member, so I chose to leave it out of this PR.Please let me know if you would like to see any or both of these limitations be solved in this PR instead.
Test Plan
call/methods.mdtest suite for method callsdescriptor_protocol.mdcall/getattr_static.mdtest suite forinspect.getattr_static