Repository navigation
BUG: Make bool(void_scalar) and void_scalar.astype(bool) consistent - #9856
Conversation
|
I get slightly different results with Python 2.7 and 1.14-dev So always True unless empty, no warnings. EDIT: Same with void array scalars. |
|
Also differs for Python 3. |
There was a problem hiding this comment.
Might want to document that this only tests scalars and scalar arrays.
|
So the rule in all these cases will be that 0D arrays and scalars containing empty byte strings, empty unicode strings, and V scalars converted to boolean yield False, and all other cases yield True. I assume this extends consistently to higher dimensional arrays. |
|
Turns out I mispoke about python 2, and hadn't tested
Can you elaborate on this? |
Yes. This was already true when converting with There's nothing special about 0d here, other than the fact that the LHS of the test |
|
Actually, the above is what the previous patch did. This patch only affects the visible behaviour of |
|
This seems correct to me, but will need a release note. |
I get
Yep, but there are tests waiting to be enabled for the others. |
You're right, I had a different execution path in my head. I've updated the top comment with correct information. |
Caused by void scalars decaying to 1d uint8 arrays before casting - `getitem` is dangerous for intermediate results Works towards numpy#9847
3e3ecfd to
931758e
Compare
|
Release note added |
|
Thanks Eric. |
The legacy `STRING/UNICODE/VOID_to_<T>` cast loops build a full array scalar for every element and hand it to the destination's setitem. For S and U sources that scalar is needlessly expensive: `PyArray_Scalar` for NPY_UNICODE allocates a scratch buffer, builds a `str`, wraps it in an argument tuple and then runs `str.__new__` on the `np.str_` subtype, which copies the data a second time. The plain bytes/str returned by getitem behaves identically in every setitem and is much cheaper. Use getitem for S and U sources, keeping `PyArray_Scalar` for VOID sources (where `void.item()` and `bool(void)` disagree, numpygh-9847, the reason the scalar was introduced in numpygh-9856) and for VOID destinations (`VOID_setitem` reads the source through the buffer protocol, which `np.str_` supports but `str` does not). Measured on 2e5 elements: `U -> int64` 3.1x faster, `U -> float64` 2.5x, `U -> bool` 6.5x, `U -> S` 3.8x, `S -> int64` 1.4x, `S -> float64` 1.35x. The reproducer in numpygh-11014 is resolved: `np.array(string_array, dtype=int)` goes from 2.2x slower than `np.array(string_array.tolist(), dtype=int)` to slightly faster. Closes numpygh-11014 Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Previously the behaviour of
void.astype(bool)was:itemsize == 0-Falseitemsize > 0-Trueitemsize == 0-False, warn about casting empty arrays to boolitemsize == 1-Trueiff the sole element is 0itemsize > 1-ValueError: setting an array element with a sequence.Compared to the behaviour of
bool(void), which is:Trueiff all bytes are zeroThe problem is that
y = x.astype(bool)is implemented asy[i] = bool(x[i].item())This patch changes it to use
y[i] = bool(x[i]).void.item()returns different results on python 2 and 3, which causes the crash.IMO,
item()is not something we should ever be using internally.Works towards #9847
This is a less objectionable subset of #9848 with no compatibility ramifications