Skip to content

Conversation

@gjoseph92
Copy link
Collaborator

@gjoseph92 gjoseph92 commented Oct 6, 2020

Follow-up/alternative to #6706.

When dispatching NumPy functions, this causes dask.Array (and its subclasses) to accept any of its parent classes in __array_function__ and __array_ufunc__, as recommended in NEP-13. This makes it easier to subclass Array, since:

  • Subclasses that don't override __array_function__/__array_ufunc__ inherit an implementation that accepts both them and plain dask.Arrays (their parent class).
  • When subclasses that do override __array_function__/__array_ufunc__ call that method on super(), it won't always delegate to the other object simply because the method was overridden—it'll only delegate if the other implements the function and isn't a parent class.

cc @jthielen

  • Tests added / passed
  • Passes black dask / flake8 dask

@gjoseph92 gjoseph92 marked this pull request as ready for review October 7, 2020 00:27
@jthielen
Copy link
Contributor

jthielen commented Oct 7, 2020

This looks good to me as an alternative to #6706 that also addresses __array_ufunc__ and binary operations. Though, if this does end up being preferred over #6706, some tests with a mock subclass would be useful here.

@gjoseph92
Copy link
Collaborator Author

@jthielen I'll add tests here specifically for binops, ufuncs, and __array_function__ on a couple toy subclasses.

But I think @shoyer is right:

I think it would be a good idea for the broader dask development team to make an intentional decision about whether subclassing dask data structures is a supported feature or not. Implementation inheritance imposes a cost in complexity that may not be worth the trouble.

It's hard to say where to draw the line on testing support for inheritance though, so I'll focus just on the things this PR is meant to address.

@TomAugspurger
Copy link
Member

Apologies for the delay in review here @gjoseph92. I'm hoping to take a look later in the week.

In the meantime, #6734 came up. Do you know if this affects it at all (at a glance, I think the answer is no)?

@jthielen
Copy link
Contributor

In the meantime, #6734 came up. Do you know if this affects it at all (at a glance, I think the answer is no)?

Thanks for the mention of that issue @TomAugspurger! I've added a comment about what I think is going on there. If it is what I think it is, then I don't believe it affects this PR.

@ncclementi
Copy link
Member

@gjoseph92 checking in here, should we merge main into this branch and see the status of CI, or is this work stale?

@gjoseph92
Copy link
Collaborator Author

Might as well merge main, I think this is still a valid change. I can tell you it's been running in production for over two years and never caused issues there :)

@github-actions github-actions bot added the array label Aug 25, 2022
@ncclementi
Copy link
Member

Ci is green after merging main. @TomAugspurger since you initially reviewed this PR, would you mind giving it a last check and if things look good we can merge it in?

@ncclementi ncclementi merged commit cf7781b into dask:main Aug 25, 2022
@gjoseph92 gjoseph92 deleted the array/dispatching/accept-superclasses branch August 25, 2022 19:36
@gjoseph92
Copy link
Collaborator Author

Thanks @ncclementi @TomAugspurger, I never thought this would end up getting in!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants