Repository navigation
Cannot create a Quantity with the new numpy.array_api #15071
Description
Description
The Array API is an effort to create a unified Array interface for many different kinds of arrays (e.g. numpy, cupy, dask, jax, etc.). A minimal (but experimental) implementation exists currently in numpy: https://numpy.org/doc/stable/reference/array_api.html. Creating a Quantity from an array with this implementation fails.
This seems to be because Quantity relies on computing the length of an array with len() but the Array API does not implement __len__.
Expected behavior
I expect to be able to create a Quantity from this unified array API. This will become important if Quantity wants to support many different array types. Targeting the Array API as opposed to any specific implementation of that standard seems like a sensible way to accomplish this. I thin this is also related to the work done in #12921 and the proposed work in astropy/astropy-project#269.
How to Reproduce
import astropy.units as u
import numpy.array_api as np
foo = u.Quantity(np.asarray([1,2,3]))---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[4], line 1
----> 1 u.Quantity(foo)
File ~/anaconda/envs/ndcube-dev/lib/python3.10/site-packages/astropy/units/quantity.py:493, in Quantity.__new__(cls, value, unit, dtype, copy, order, subok, ndmin)
490 if unit is None:
491 unit = value_unit # signal no conversion needed below.
--> 493 elif isiterable(value) and len(value) > 0:
494 # Iterables like lists and tuples.
495 if all(isinstance(v, Quantity) for v in value):
496 # If a list/tuple containing only quantities, convert all
497 # to the same unit.
498 if unit is None:
TypeError: object of type 'Array' has no len()Versions
macOS-10.16-x86_64-i386-64bit
Python 3.10.12 (main, Jul 5 2023, 15:34:07) [Clang 14.0.6 ]
astropy 5.3.1
Numpy 1.25.1
pyerfa 2.0.0.3
Scipy 1.11.1
Matplotlib 3.7.2
Activity
pllim commentedon Jul 15, 2023
Thanks! I don't think we ever promised to follow this Array API, so I changed the label to feature request.
Maybe #13460 is related?
mhvk commentedon Jul 16, 2023
As
Quantityis a subclass ofndarray, I think it will ever only be as compliant asndarrayis. That said, there is no reason for us not to support the array api on input. In this case, it seems the real problem is that the input is not recognized as already being an array - the stanza that fails right now is for lists and tuples where one has no choice but to uselen(aside: it does seem strange to be iterable and not provide a__len__!). PR very welcome.p.s. Longer term it would be nice if
Quantitywere agnostic to how the array below it is stored -- but that is a much larger effort.wtbarnes commentedon Jul 17, 2023
Ah I see. I didn't catch that. Presumably this is due to an instance check on
np.arrayspecifically?I am not sure what the exact motivation for omitting this is. However, calling
lenon a NumPy array returns only the length of the first dimension which is potentially confusing/ ambiguous. Array` does have size and shape attributes though.Definitely agree! I wonder if using the array-api-compat wrapper would be a good solution in lieu of a larger refactor of Quantity? https://github.com/data-apis/array-api-compat
I should apologize as I think my initial post was a bit terse. I fully realize this is not a simple issue and intersects with a number of ongoing efforts in rethinking Quantity, including support for things like Dask arrays. I think targeting the Array API (which seems to be maturing rapidly!) is the right direction though certainly a large effort.
I'm happy to submit a PR to try to fix this particular issue, but I suspect it's really only a symptom and I am not overly familiar with Quantity internals (For context, I landed here by first attempting to make ndcube Array API compatible and quickly ran into this issue). Maybe a good first step is just seeing where the tests fail by replacing
import numpy as npwithimport numpy.array_api as np.mhvk commentedon Jul 17, 2023
Since we are an
ndarraysubclass, we have to treatndarraydifferently (we can just use.view()), but there would definitely be a case for at least recognizing anyarray_apiarray and, for now, turning those inndarrayand via that intoQuantity.Another question is to what extent
Quantity's behaviour follows thearray_apisufficiently that it is correctly passed through compliant libraries.Anyway, all interesting (if not trivial to solve); thanks for bringing it up!
[-]Cannot creat a `Quantity` with the new `numpy.array_api`[/-][+]Cannot create a `Quantity` with the new `numpy.array_api`[/+]olintoeduardo commentedon Oct 24, 2023
Hi, I would like to contribute to this issue, is that possible? Thanks
mhvk commentedon Oct 24, 2023
@olintoeduardo - yes, definitely possible. Good to start with adding a test case and then adjust
Quantity.__new__so things get recognized properly.p.s. We are very close to feature freeze so won't have too much time to comment/help, etc.
MridulS commentedon Dec 21, 2023
This is fixed by #15752
mhvk commentedon Dec 21, 2023
Indeed it is, thanks again! Closing this issue.