NEP: Adjust NEP-35 to make it more user-accessible - #17093

This section is wonderful, thank you @pentschev for writing it. =) Definitely exactly what I was looking for. =) The only thing I think would be useful is to have a complete usage example, "eg here is scikit-learn k-means now — without any option to create like, even if it works with a CuPy array as input it will end up with a NumPy array as output. But after this NEP and modifying the code to look like below, it will work completely within CuPy, or dask, or other arrays implementing this protocol."

I was trying to rack my brain for examples in scikit-image that need this, but can't come up with any off the top of my head. Happy to go hunting for one, but I figure there might be existing examples that motivated this NEP in the first place and that would be simple enough to be included?

Update: my_pad below is sufficient. I wonder whether it belongs in this section rather than the next one, ie move up to line 113 (cupy's TypeError) into this section, then start the next section.

This section is wonderful, thank you @pentschev for writing it. =) Definitely exactly what I was looking for. =)

To be fair this is not a new addition, it was already in the original text, it just got moved a bit. See https://numpy.org/neps/nep-0035-array-creation-dispatch-with-array-function.html#usage-guidance .

Update: my_pad below is sufficient.

I'm not sure if you mean that my_pad already addresses the "complete usage example" you mentioned above or the strikethrough text. Are you confirming the existing example suffices?

I wonder whether it belongs in this section rather than the next one, ie move up to line 113 (cupy's TypeError) into this section, then start the next section.

I tried to follow the NEP X -- Template and Instructions. To my understanding, it should belong in the "Motivation and Scope" section, where it is now.

"Motivation and Scope: ... It should describe the existing problem, who it affects, what it is trying to solve, and why."

Yes, my_pad is sufficient, and I agree, it belongs in motivation and scope, but it is currently in Usage, and I was asking whether it should be moved to motivation and scope.

Yes, my_pad is sufficient, and I agree, it belongs in motivation and scope, but it is currently in Usage, and I was asking whether it should be moved to motivation and scope.

Are we still talking about the paragraph saying "We expect that this functionality will be mostly useful to library developers, ..."? That is in Motivation and Scope already, not in Usage and Impact.

This was one of the reasons why I got confused in the beginning. "Why should I even be able to create other arrays if I am already using those libs? I can create via CuPy or Dask whatever I need".

That detail is being lost with this compact narrative.

This was suggested in #17093 (comment) . Given that the intent was to make the NEP clearer to users, I agree with @ilayn that the detail is getting lost.

This begs the question "Why can't I just use the my_pad from earlier on dask arrays?".

That is what is explained in https://github.com/numpy/numpy/pull/17093/files#diff-e62e74367c379dd1a3b21bc549c1eba6R166-R173 .

So to be clear:

• my_pad(dask_array) should work just fine, thanks to the like argument and __array_function__
• my_dask_pad(dask_array) is an example of what code internal to dask might look like.

Mentioning my_pad without comparing it to my_dask_pad makes reading this a little confusing to me

At any rate, it seems that meta_from_array is redundant here. Perhaps a more convincing example would be something like

# disclaimer: I have never used dask, I am guessing the API from your summary!
def pad_chunks(arr: dask.array, padding: ArrayLike):
    # we could use `np.array(padding, like=arr).chunks[0]`, but this is more direct
    padding = np.array(padding, like=meta_from_array(arr))
    return dask.from_chunks([padding] + arr.chunks + [padding])

where the key point is you don't actually need meta_from_array except:

• Inside dask.array.__array_function__
• As an optimization inside dask-specific functions.

my_pad(dask_array) should work just fine, thanks to the like argument and array_function

It would work fine for a "pure" Dask array, that's backed by NumPy. It won't work fine for a CuPy-backed Dask array, which needs to be handled accordingly, thus the meta_from_array call.

meta_from_array would be redundant if a Dask Array wasn't and array wrapping another type of array, which is why we need to handle it properly.

Anyway, I tried improving the text in 57f78df , can you check if it makes things clearer?

This code would work just fine without the second line. To motivate it, perhaps:

This code would work just fine without the second line.

I don't get this comment, are you saying it works without the np.concatenate?

The naive reader may not understand why the call to padding = np.array(padding, like=arr) is needed.

.. but then it is explained in the text below.