-
-
Notifications
You must be signed in to change notification settings - Fork 12k
Description
With the various different array libraries in place, it's gotten a bit tricky to write general code that works for all of them. Previously we might checking if some array is an instance of a NumPy ndarray, but that doesn't generalize. One solution people have come up with are "duck" checks where various attributes like shape, dtype, etc. are checked. This can work or can run into issues depending on whether these checks are representative of what the user is looking for.
In particular we have some arrays that are concrete (like NumPy, CuPy, Sparse, etc.). These arrays own memory and can easily manipulate values they contain or perform IO. Other arrays are abstract (like Dask, Xarray, etc.). They indirectly hold data through other objects and are not always able to manipulate their contents easily. Serialization for these arrays is an orchestration of serialization of their contents or perhaps their state depending.
It's useful to understand when one has a concrete array vs. an abstract array. However we lack a mechanism to do that today. Am raising this issue so that we can discuss how we might solve this.