Repository navigation
array(...) casts mix lists of int and string to string automatically (excepted dtype object) #6550
Description
The following code explains the issue
>>> import numpy
>>> print numpy.__version__
1.10.1
>>> a = [["a", 1], [2, 3]]
>>> b = numpy.array(a)
>>> print repr(a)
[['a', 1], [2, 3]]
>>> print repr(b)
array([['a', '1'],
['2', '3']],
dtype='|S1')
I would have expected that not specifying a dtype would have given the result as if I had specify a dtype(object) (to be able to hold both ints and strings)
>>> b = numpy.array(a, dtype=numpy.dtype(object))
>>> print repr(b)
array([['a', 1],
[2, 3]], dtype=object)
dtype : data-type, optional
The desired data-type for the array. If not given, then the type will be determined as the minimum type required to hold the objects in the sequence. This argument can only be used to ‘upcast’ the array. For downcasting, use the .astype(t) method.
But I may have missed some point in the docs
Reacted by kalvdans
Activity
sdementen commentedon Oct 22, 2015
I think the same bug is behind the weird behavior of assert_array_equal here below
shoyer commentedon Oct 22, 2015
I agree with you than object dtype would be preferable, but this is also pretty long standing behavior in numpy.
sdementen commentedon Oct 22, 2015
The bug is only with strings and float ? Or on any mixed type data ?
In any case, it could be good to fix the documentation with the explanation
of how the dtype is defined in these cases.
On Thursday, October 22, 2015, Stephan Hoyer [email protected]
wrote:
njsmith commentedon Oct 22, 2015
This smells like it's related to gh-6061 (which has an unreviewed PR sitting around since July: gh-6067). Merging gh-6067 directly may not fix this because I'm not sure that
arrayuses the same promotion rules as ufuncs, but the same basic logic applies. Silently "upcasting" numbers into strings doesn't make any sense, and I think the only reason it's survived this long is that people don't use numpy strings much so they don't get as much scrutiny as other things.njsmith commentedon Oct 22, 2015
@sdementen: Though note that there is also an independent plan to change
arrayso that it never automatically returns object dtypes and requires you to specifydtype=objectif that's what you want. So specifyingdtype=objectexplicitly as a workaround for this issue is also future-proofing your code...sdementen commentedon Oct 22, 2015
I felt upon this bug while using xlwings (interface with Excel). It was
reading a range of cells that contained strings (header of the data) and
numbers (the data itself). It used a numpy array to read everything to be
able to use the nice slice notation of arrays. However, as there were
strings in the header, it converted the whole array to strings. Slicing
then to get the header gave strings (which is ok) but slicing to get the
data returned also strings (not ok).
Converting always to dtype=object is neither optimal as sometimes the range
of cells contains only numbers (for which object is not optimal as dtype).
But indeed, the bug can be circumvented from outside numpy.
On Thursday, October 22, 2015, Nathaniel J. Smith [email protected]
wrote:
chrisjbillington commentedon May 11, 2018
This one has tripped me up recently - putting tuples in HDF5 attributes via h5py converts them to numpy arrays, and I put a tuple of an int and a string in and got back two strings later.
Most of the casting numpy does is to types that pass equality checks, but since
'1' == 1isFalse, this casting has extra risk, and I would definitely have preferred an error. It might be long standing behaviour in numpy, but I'd be happy to see it deprecated.Al2a2d2m commentedon Nov 16, 2018
I dont this that this is a bug. we can find the same behavior in R. both upcast dtype to the minimal type required to handle data. object dtype seems extreme to me. for example numpy uses it when we do not have a matrix structure that is array elements did not have the same dimemsion ex: np.[1,2,3],[1,2]])