"setting an array element with a sequence" error could be improved.

This is a bit of a nitpick, but I think this would improve user-friendlyness.
Currently np.asarray([[1, 2], [2, 3, 4]], dtype=np.float) yields "setting an array element with a sequence."

I think it would be more helpful if it yielded something like "Can't create array from lists. Column length %d of column %d doesn't match column length %d."

I imagine the error bubbles up from somewhere deep in the code and makes perfect sense where it is raised. It is just something that comes up a lot, and a more friendly and / or informative message could be helpful. So maybe we could catch it further up and include more information if possible.

This is a duplicate of #5303.

indeed, sorry for the noise.

Re-opening in light of #5303 (comment).

There are a few different but related issues. One issue is explicit vs. implicit dtype=object array creation. The github issue for that is #5353 (also #6070, I see). Another issue is the possibility to create ndarrays of objects with conformable shapes without automatically peeking into those objects and trying to figure out what the user wants. That issue is #5933.

We could keep this issue open, for improving the error message for specific cases like np.asarray([[1, 2], [2, 3, 4]], dtype=np.float). If the error message could be special-cased for np.asarray(a, dtype=b) even for really specific kinds of a and b, this could be useful. For example, maybe a special case could apply when a is a Python list recursively containing more lists and numeric values (and maybe only when the all leaves of the tree defined by the list all share the same depth?) and where b is some simple numeric dtype.

Here are a few more cases that give the same error message, but which may or may not be in the scope of this issue.

>>> np.array([[[1, 2], [3, 4], [5, 6]], [[1], [2], [3]]], dtype=int)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: setting an array element with a sequence.

>>> np.array([[1, 2, 3], 4], dtype=int)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: setting an array element with a sequence.

>>> np.array([(1, 2), (2, 3, 4)], dtype=int)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: setting an array element with a sequence.

Thanks for your summary. I think we are on the same page.
Btw, my motivation was that this might be inside a library, and it might be very non-obvious to the user why a line is failing. I probably won't have time to work on this right now :-/

Here's an analogous example with structured arrays, giving a different error message.

>>> np.array([[(1, 1), (2, 2)], [(2, 2), (3, 3), (4, 4)]], dtype=[('foo', '>i4'), ('bar', '>i4')])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: expected a readable buffer object

When the lists are conformable there is no error.

>>> np.array([[(1, 1), (2, 2), (3, 3)], [(2, 2), (3, 3), (4, 4)]], dtype=[('foo', '>i4'), ('bar', '>i4')])
array([[(1, 1), (2, 2), (3, 3)],
       [(2, 2), (3, 3), (4, 4)]], 
      dtype=[('foo', '>i4'), ('bar', '>i4')])

I wanna bump this in sight of the BoF in terms of directions. I think better error messages in general would be great, with this my "favorite" examples.

@amueller

Thanks @amueller. This has 31 thumbs up, clearly valuable - tagging for the next release to make sure it doesn't drift down the list again.

A new issue/strategy for better error messages in general would also be useful. Could be good to discuss during the sprint.

I tried to think about others and/or come up with general strategies but drew a blank. Maybe looking at highly voted stack overflow issues might help to find common problems? (Gonna do that for sklearn right now lol).

Hmm most errors are install errors or this one lol https://stackoverflow.com/search?q=%5Bnumpy%5D+error&tab=votes

@stefanv

Might be good to discuss this at next BIDS meeting. @stefanv, @rgommers, Ping.

I removed the 1.18.0 milestone. We're not forgetting about this! There are new designs in progress that will affect this issue, but they will not be implemented for 1.18.

I have a branch, where the first example will give this:

In [1]: np.asarray([[1, 2], [2, 3, 4]], dtype=np.float64)                                                               
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

I could try to make it more useful, any suggestions?

Thanks for working on this, this is already much better!

Do you keep the first part for backward compatibility? It's not really meaningful to a user, right?
I'm not sure how tricky it would be but I think the ideal would be to give the first inhomogeneous part, say "shape (2,) in [1] and shape(3,) in [1]" (or some much better phrasing of this).

Yeah, I thought so, plus there are some stranger paths, where we give this error but do not have the context. Although, I suppose there may not be a need to align those, because in those other paths the message is more logic, because it ends up being things like arr[0] = [1,2,3]

Thats true, that would be nice. I could give the first inhomogeneous part, but it may be hard to report the element where the mismatch occurred. Maybe something for later, rephrasing I can do now. I also have a "failed shape", but I would need two values for the mismatched dimension, and ideally know where exactly the mismatch was.

I won't look at it right now, but I think after my PR is in, it may be possible (or at least easier) to expand the message!

I think this is already a big improvement, there's definitely a trade-off in how much detail to provide.

I am going to close the issue now, since gh-16200 is in, and the error is now better. But, I admit there is probably still a lot of room for improvement (if just rephrasing), and that is now much easier to do, so I invite contributions or suggestions!

I have a 3-D numpy array and im getting the same error if i try to change its dataype to float. Also, if i try using the same data without changing anything with sklearn I get the same error while if I use it with keras, I get ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type numpy.ndarray)..
I am trying to do a machine learning task and my current dataset is a numpy array of shape (556,1) and for every sample it have 27 features so (27,1) and each feature has variable length and the same feature might have a different shape for different samples.
Any help on how I get fix this?

Navigation Menu

Repository navigation

"setting an array element with a sequence" error could be improved. #6584

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

"setting an array element with a sequence" error could be improved. #6584

Description

Activity

argriffing commented on Oct 29, 2015

amueller commented on Oct 29, 2015

argriffing commented on Oct 30, 2015

amueller commented on Oct 30, 2015

argriffing commented on Oct 30, 2015

amueller commented on Jul 12, 2018

rgommers commented on Jul 12, 2018

amueller commented on Jul 12, 2018

amueller commented on Jul 12, 2018

charris commented on Nov 17, 2018

WarrenWeckesser commented on Oct 15, 2019

seberg commented on Jun 17, 2020

amueller commented on Jun 17, 2020

seberg commented on Jun 17, 2020

amueller commented on Jun 17, 2020

seberg commented on Jul 9, 2020

bhavaygg commented on Sep 20, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions