Repository navigation
BUG: array's append should respect the type of the array you are appending to #22427
Description
Describe the issue:
Calling np.append on a float32 array will return a float64 array unless we append explicitly a float32.
It seems confusing (and a potential performance issue) that the entire array gets casted up just because I appended something.
I think the type of the original array should be preserved, or append be given an optional dtype argument like other methods have.
Reproduce the code example:
import numpy as np
a = np.array([1, 2], dtype=np.float32)
print(a.dtype) # prints float32
b = np.append(a, 42)
print(b.dtype) # prints float64The workaround is to explicitly append a float32 by doing np.float32(42) first, but it's quite hard to find out this cast is happening in the first place.
NumPy/Python version information:
1.23.3 3.10.6 (main, Aug 30 2022, 04:58:14) [Clang 13.1.6 (clang-1316.0.21.2.5)]
Context for the issue:
- Apple's mps only supports float32.
- numpy defaults to float64 everywhere if you are not careful.
- This causes a crash on Macs (eg: when using numpy arrays from pytorch), but it could be a hidden performance issue on other places because of the casts (which IMO would be worse than the crash).
Activity
maidaly commentedon Oct 12, 2022
I solved it by adding dtype parameter to the append function to make sure that the new copy array generated from the append has the wanted dtype ..... and here is the result.
seberg commentedon Oct 13, 2022
Hmmm, adding a
dtypekwarg does add a decent work-around. I do agree that in principle we should probably just use thearrdtype and it is unfortunate that this is not the case.Should we actually consider changing this, though?
appendis a very thin alias aroundconcatenateafter all. So if we added aFutureWarning, users could switch toconcatenateif they want the old behavior.If they need to silence the warning and get the new behavior, they would have to cast manually unfortunately.
np.insertdoes have the "expected" behavior of casting to the first array.EDIT: I am very much presuming that most users already get the first array's
dtypein any case, so theFutureWarningshouldn't be ubiquitous.MatteoRaso commentedon Oct 13, 2022
I think the change is good. The append function is far more commonly used than the concatenate function, so it would be annoying for users to constantly get a bunch of warnings in their code.
mattip commentedon Oct 13, 2022
The documentation for append does not mention concatenate. It makes sense to me to add a
dtypeand even acastargument to append, and to change the documentation to mention append is a thin layer around concatenate. What would the proposed message of a FutureWarning be?seberg commentedon Oct 13, 2022
The warning would be something like: