Skip to content

Checking array types (concrete vs. abstract) #14891

@jakirkham

Description

@jakirkham

With the various different array libraries in place, it's gotten a bit tricky to write general code that works for all of them. Previously we might checking if some array is an instance of a NumPy ndarray, but that doesn't generalize. One solution people have come up with are "duck" checks where various attributes like shape, dtype, etc. are checked. This can work or can run into issues depending on whether these checks are representative of what the user is looking for.

In particular we have some arrays that are concrete (like NumPy, CuPy, Sparse, etc.). These arrays own memory and can easily manipulate values they contain or perform IO. Other arrays are abstract (like Dask, Xarray, etc.). They indirectly hold data through other objects and are not always able to manipulate their contents easily. Serialization for these arrays is an orchestration of serialization of their contents or perhaps their state depending.

It's useful to understand when one has a concrete array vs. an abstract array. However we lack a mechanism to do that today. Am raising this issue so that we can discuss how we might solve this.

cc @mrocklin @pentschev @rgommers @shoyer

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions