Skip to content

Comments

[red-knot] Support super#17174

Merged
carljm merged 31 commits intoastral-sh:mainfrom
cake-monotone:cake/support-super
Apr 16, 2025
Merged

[red-knot] Support super#17174
carljm merged 31 commits intoastral-sh:mainfrom
cake-monotone:cake/support-super

Conversation

@cake-monotone
Copy link
Contributor

Summary

closes #16615

This PR includes:

  • Introduces a new type: Type::BoundSuper
  • Implements member lookup for Type::BoundSuper, resolving attributes by traversing the MRO starting from the specified class
  • Adds support for inferring appropriate arguments (pivot_class and owner) for super() when it is used without arguments

When super(..) appears in code, it can be inferred into one of the following:

  • Type::Unknown: when a runtime error would occur (e.g. calling super() out of method scope, or when parameter validation inside super fails)
  • KnownClass::Super::to_instance(): when the result is an unbound super object or when a dynamic type is used as parameters (MRO traversing is meaningless)
  • Type::BoundSuper: the common case, representing a properly constructed super instance that is ready for MRO traversal and attribute resolution

Terminology

Python defines the terms bound super object and unbound super object.

An unbound super object is created when super is called with only one argument (e.g.
super(A)). This object may later be bound via the super.__get__ method. However, this form is rarely used in practice.

A bound super object is created either by calling super(pivot_class, owner) or by using the implicit form super(), where both arguments are inferred from the context. This is the most common usage.

Follow-ups

  • Add diagnostics for super() calls that would result in runtime errors (marked as TODO)
  • Add property tests for Type::BoundSuper

Test Plan

  • Added mdtest/class/super.md

@MichaReiser MichaReiser added the ty Multi-file analysis & type inference label Apr 3, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 3, 2025

mypy_primer results

No ecosystem changes detected ✅

@cake-monotone

This comment was marked as outdated.

@AlexWaygood
Copy link
Member

Most of the diffs in the mypy primer seem to be caused by the lack of support for Self

This one is a crash:

itsdangerous (https://github.com/pallets/itsdangerous)
+ 
+ thread '<unnamed>' panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/d758691/src/function/fetch.rs:154:25:
+ dependency graph cycle querying all_negative_narrowing_constraints_for_expression(Id(c413)); set cycle_fn/cycle_initial to fixpoint iterate
+ note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
+ Rayon: detected unexpected panic; aborting

@cake-monotone
Copy link
Contributor Author

OMG 😅 I’ll check the panic in the mypy primer first and then request review again.

@cake-monotone cake-monotone marked this pull request as draft April 3, 2025 13:04
@MichaReiser
Copy link
Member

I'm not sure if the panic is due to your PR. It is the same that @carljm has seen on other mypy primer projects.

@AlexWaygood
Copy link
Member

AlexWaygood commented Apr 3, 2025

I'm not sure if the panic is due to your PR. It is the same that @carljm has seen on other mypy primer projects.

It shows that this PR causes us to panic on code that we did not previously panic on. Maybe that's acceptable if we want to defer adding cycle handling to the relevant function for now, but it's going to lead to pretty confusing diffs if we include projects we panic on in mypy_primer. I think at the very least we should remove the project from the list in the workflow file of projects we run on in CI if we're going to start panicking on that project:

--project-selector '/(mypy_primer|black|pyp|git-revise|zipp|arrow|isort|itsdangerous|rich|packaging|pybind11|pyinstrument|typeshed-stats|scrapy)$' \

but also: adding cycle handling to a query probably isn't that difficult, and it's nice to at least not increase the amount of code we panic on. @cake-monotone, you can see an example of how to do it here: #16958 :-) I'd imagine adding cycle handling to all_negative_narrowing_constraints_for_expression probably won't look too different to the changes I made in that PR.

@MichaReiser
Copy link
Member

Agree. @carljm might be able to prioritize the fix real quick to unblock this PR

@cake-monotone
Copy link
Contributor Author

cake-monotone commented Apr 3, 2025

Agree. @carljm might be able to prioritize the fix real quick to unblock this PR

It's okay! I'll try to fix it soon...!

I'm not sure if the panic is due to your PR. It is the same that @carljm has seen on other mypy primer projects.

It shows that this PR causes us to panic on code that we did not previously panic on. Maybe that's acceptable if we want to defer adding cycle handling to the relevant function for now, but it's going to lead to pretty confusing diffs if we include projects we panic on in mypy_primer. I think at the very least we should remove the project from the list in the workflow file of projects we run on in CI if we're going to start panicking on that project:

--project-selector '/(mypy_primer|black|pyp|git-revise|zipp|arrow|isort|itsdangerous|rich|packaging|pybind11|pyinstrument|typeshed-stats|scrapy)$' \

but also: adding cycle handling to a query probably isn't that difficult, and it's nice to at least not increase the amount of code we panic on. @cake-monotone, you can see an example of how to do it here: #16958 :-) I'd imagine adding cycle handling to all_negative_narrowing_constraints_for_expression probably won't look too different to the changes I made in that PR.

I just need to define a custom cycle_recover function and cycle_initial for all_negative_narrowing_constraints_for_expression, right?
If I’m understanding that correctly, it shouldn’t take too long

@AlexWaygood
Copy link
Member

I just need to define a custom cycle_recover function and cycle_initial for all_negative_narrowing_constraints_for_expression, right? If I’m understanding that correctly, it shouldn’t take too long

I think that's correct, yes!

@cake-monotone
Copy link
Contributor Author

cake-monotone commented Apr 3, 2025

Ah... it's still not working. the mypy primer still panics 😢

Here's the commit if you'd like to take a look:
cake-monotone@a737bfe

@AlexWaygood
Copy link
Member

hmm, okay, we may need @carljm's advice on how to fix this; he's the fixpoint expert :-)

@cake-monotone

This comment was marked as outdated.

@cake-monotone cake-monotone marked this pull request as ready for review April 3, 2025 14:18
@cake-monotone
Copy link
Contributor Author

It turns out that there were actually two functions causing the panic: all_narrowing_constraints_for_expression and all_negative_narrowing_constraints_for_expression. After applying what Alex suggested to both of them, the panic was resolved.

@carjim Hmm, just checking, are you planning to open a separate PR to implement cycle_fn to these functions?
If not, I’d be happy to include it in this PR :)

Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest fixing the cycles as a separate PR

@cake-monotone cake-monotone marked this pull request as ready for review April 13, 2025 07:38
@cake-monotone cake-monotone marked this pull request as draft April 13, 2025 07:39
@cake-monotone
Copy link
Contributor Author

cake-monotone commented Apr 13, 2025

Sorry for the long delay on this PR. I've been pretty drained over the past two weeks due to some other things going on, and didn’t have the energy to polish it to a state I was happy with.

Anyway, since the last review, a lot has changed — including improvements in handling dynamic types, fixes for incorrect enclosing class inference, and support for generic classes.

I'd really appreciate it if you could take another look. :)

@cake-monotone cake-monotone marked this pull request as ready for review April 13, 2025 11:31
@cake-monotone cake-monotone requested a review from carljm April 13, 2025 11:31
Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fantastic! Just a few nits. I pushed fixes for some of these since I'd already made the changes locally for testing. I think the main thing left to address is more tests for use of super with generics, and addressing the debug assertion that I don't think is correct. Also looks like it needs a rebase and to address a conflict in types.rs. Please let me know if you don't have time to address this and I can find some time! I want to get this landed and not have it accumulate more conflicts.

@carljm carljm enabled auto-merge (squash) April 16, 2025 18:39
@carljm
Copy link
Contributor

carljm commented Apr 16, 2025

Excellent work on this PR, thank you!!

@carljm carljm merged commit 649610c into astral-sh:main Apr 16, 2025
22 checks passed
dcreager added a commit that referenced this pull request Apr 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[red-knot] support super

4 participants