Repository navigation
Unable to test equality of structured dtypes with no fields #13438
Description
If a structure type has no fields, equality throws
>>> a = np.zeros(10, dtype=[])
>>> a == a
ValueError: No fields found.Original report:
Test for equality of empty masked arrays raises an exception.
Reproducing code example:
In [1]: import sys, numpy; print(numpy.__version__, sys.version)
1.16.3 3.6.1 (default, May 18 2017, 16:32:13)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]
In [2]: a = numpy.ma.masked_array([], dtype=[], masked=False) # or masked=True
In [3]: a == NoneError message:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-3-991e7ea0a64c> in <module>
----> 1 a == None
~/.../lib/python3.6/site-packages/numpy/ma/core.py in __eq__(self, other)
4043 and other considered equal only if both were fully masked.
4044 """
-> 4045 return self._comparison(other, operator.eq)
4046
4047 def __ne__(self, other):
~/.../lib/python3.6/site-packages/numpy/ma/core.py in _comparison(self, other, compare)
3996 # Now take care of the mask; the merged mask should have an item
3997 # masked if all fields were masked (in one and/or other).
-> 3998 mask = (mask == np.ones((), mask.dtype))
3999
4000 else:
ValueError: No fields found.Numpy/Python version information:
1.16.3 3.6.1 (default, May 18 2017, 16:32:13)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]
Activity
seberg commentedon May 1, 2019
Uggh, should structured data types with no fields even be allowed?
In any case, the error is somewhat correct, at least not specific to masked arrays. While it does not happen for arrays, that is only because we replace the error with
Falsecurrently (and we may at some point change that, its been long deprecated, but I think there have been opinions against actually going through with it).Frankly, I am not sure we should take action here. Out of curiosity do you actually have a use case for this dtype?
[-]Unable to test equality of empty masked arrays[/-][+]Unable to test equality of structured dtypes with no fields[/+]eric-wieser commentedon May 1, 2019
Updated the top-post and title to reflect that masked arrays are irrelevant here.
I'd argue this should just return
True, in the same way that() == ()is True.seberg commentedon May 1, 2019
@eric-wieser yes, sure (or an array of True). What I am not sure about is, what the point of a no field structured type is. On the other hand, I guess there is also no real reason against just allowing it.
mcara commentedon May 1, 2019
I honestly do not know, but currently you do and they crash, occasionally. This issue arose from astropy/astropy#8647 where I am trying to make
Table.as_array()to return an empty array when a table is empty - for consistency purpose. When the table is masked (btw, sorry for carrying over masking from tables to the title of this issue), it creates an empty structured array. For this type of array testing for equality to something crashes.I do not disagree with you that this is a rare situation but I think we should try to be consistent in how we handle edge cases. There are a couple of places in
astropyor affiliated packages when a function may return an empty table (based on my memory - I may be wrong). Maybe we can do things differently if this is not a supported usage of arrays. But my first impression was that if an empty array is allowed to be created then why couldn't I use it for comparison.seberg commentedon May 1, 2019
Yeah, I was a bit misguided, there is definitely no point in not allowing this (especially going backward!), so the void comparison code really should just be capable of handling this.
eric-wieser commentedon Jun 8, 2019
This unfortunately is difficult to do correctly - not only does the shape of this array have to be correct, but it should probably inherit the right type through the array_ufunc mechanism.
I think this issue might be unfixable until void comparison is done via a ufunc.
seberg commentedon Jun 9, 2019
Well, doing void comparison via a ufunc is very much doable though. And . while string ufuncs would be a bit annoying, a string comparison ufunc should be very much doable. Right now I would prefer not to put it on the agenda though probably. 1.17 is almost there, and I have some hope that we have some changes until 1.18... OTOH, the loops itself would still all work fine I suppose.
BUG: Fix fieldless comparison broadcasting
mattip commentedon Sep 8, 2019
Confirm that #14393 closed this: