Repository navigation
Unexpected behaviour by numpy.pad() with mode="linear_ramp" and uint dtypes in 1.17 #14191
Description
Since 1.17 numpy.pad() with mode="linear_ramp" seems to not use the array's edge value as the end value, but instead the value range maximum of the array's dtype. This seems to happen only for uint dtypes and only if end_value > edge_value.
Reproducing code example:
import sys, numpy as np; print(np.__version__, sys.version)
for dtype in ["uint8", "uint16", "uint32",
"int8", "int16", "int32",
"float16", "float32", "float64"]:
arr = np.zeros((1,), dtype=dtype)
arr_pad = np.pad(arr, (3, 0), mode="linear_ramp", end_values=3)
print("%07s: %s" % (dtype, arr_pad))Output for 1.17:
1.17.0 3.7.3 | packaged by conda-forge | (default, Jul 1 2019, 21:52:21)
[GCC 7.3.0]
uint8: [ 3 87 172 0]
uint16: [ 3 21847 43692 0]
uint32: [ 3 1431655767 2863311532 0]
int8: [3 2 1 0]
int16: [3 2 1 0]
int32: [3 2 1 0]
float16: [3. 2. 1. 0.]
float32: [3. 2. 1. 0.]
float64: [3. 2. 1. 0.]Output for 1.16.4 (on a python 2.7 system):
('1.16.4', '2.7.15 |Anaconda, Inc.| (default, Oct 23 2018, 18:31:10) \n[GCC 7.3.0]')
uint8: [3 2 1 0]
uint16: [3 2 1 0]
uint32: [3 2 1 0]
int8: [3 2 1 0]
int16: [3 2 1 0]
int32: [3 2 1 0]
float16: [3. 2. 1. 0.]
float32: [3. 2. 1. 0.]
float64: [3. 2. 1. 0.]Further Observations
Using a 2D array does not fix the issue.
Using end_values=(3, 0) instead of end_values=3 does not fix the issue.
However, the issue does not appear in 1.17 so long as the edge value is >=3:
import sys, numpy as np; print(np.__version__, sys.version)
for dtype in ["uint8", "uint16", "uint32",
"int8", "int16", "int32",
"float16", "float32", "float64"]:
arr = np.zeros((1,), dtype=dtype)
arr = arr + 3
arr_pad = np.pad(arr, (3, 0), mode="linear_ramp", end_values=(3, 0))
print("%07s: %s" % (dtype, arr_pad))Output:
1.17.0 3.7.3 | packaged by conda-forge | (default, Jul 1 2019, 21:52:21)
[GCC 7.3.0]
uint8: [3 3 3 3]
uint16: [3 3 3 3]
uint32: [3 3 3 3]
int8: [3 3 3 3]
int16: [3 3 3 3]
int32: [3 3 3 3]
float16: [3. 3. 3. 3.]
float32: [3. 3. 3. 3.]
float64: [3. 3. 3. 3.]Related
Maybe this is related to issue #12293?
There is also a paragraph in the 1.17 release notes saying:
Additionally, there are some corner cases with behavior changes:
Padding max < min has changed to be more consistent across dtypes, but should not be relied upon.
This is in a section about changes to clip, but maybe it is still somehow related?
Activity
seberg commentedon Aug 5, 2019
@lagru seems like we missed some corner case here, do you wan to have a look? Otherwise I can.
lagru commentedon Aug 6, 2019
@seberg Thanks for the summoning.
@aleju Thank you for the detailed bug report and the time you took investigating! I can reproduce this on 1.18.0.dev0+5f7f8e1 (3.7.3). This line
numpy/numpy/lib/arraypad.py
Line 73 in d9b1e32
is the culprit. In case the padded array is an unsigned type and the difference is negative we get an overflow because
stophas the same type as the padded array. In case the edge value is >= 3 the difference is positive again and everything works as intended.Simply casting
stopto a float64 does fix this. Not sure if that is the most elegant/efficient way but I'll add a PR where this can be discussed.lagru commentedon Aug 6, 2019
I'm kind of surprised that this is discovered only now. A similar bug (switching edge and end_value) affects earlier versions as well. E.g. in 1.16:
Does this mean that either "linear_ramp", unsigned integers or both are only used infrequently?
lagru commentedon Aug 6, 2019
1.16 is still actively maintained, isn't it? That branch would need a patch as well.
By the way, is there an official place to look which NumPy versions are still maintained? I couldn't find one.
seberg commentedon Aug 6, 2019
Thanks for the heads up, Chuck already labeled the PR accordingly. We usually do not backport more than one release. 1.16 is the exception, since it will see at least one more bugfix release. My guess is it will not be supported longer then 1.17.x (unless someone outside the current team steps up). It is possible that there will be only one more bugfix release in 1.16.x series.
charris commentedon Aug 6, 2019
I'm going to do a 1.16.5 soon, and maybe one more after that, depending. If a (different) patch is needed, sooner rather than later would be better.
charris commentedon Aug 6, 2019
Note to self: may need 1.16.x backport as well. I wish there could be multiple milestones :)
BUG: Simplify linear ramp padding with linspace
BUG: Replace _linear_ramp with linspace
BUG: Replace _linear_ramp with linspace