Repository navigation
NumPy's implementation of scalar integer ** integer has very weird casting #7449
Description
Apropos #7447 (CC @ewmoore @charris):
I was curious about what was going on here, and it looks much uglier than we realized :-(
Little test script that tries doing 4 ** -4 using different types:
Old testing script (and results)
signed_int_types = [np.int8, np.int16, np.int32, np.int64, int]
all_int_types = signed_int_types + [np.uint8, np.uint16, np.uint32, np.uint64]
for base_type in all_int_types:
for exponent_type in signed_int_types:
result = base_type(4) ** exponent_type(-4)
print("{} ** {} -> {} ({})".format(base_type.__name__,
exponent_type.__name__,
result.__class__.__name__,
result))On current master (after #7447 is merged):
int8 ** int8 -> float32 (0.00390625)
int8 ** int16 -> int16 (0)
int8 ** int32 -> int32 (0)
int8 ** int64 -> int64 (0)
int8 ** int -> int64 (0)
int16 ** int8 -> float32 (0.00390625)
int16 ** int16 -> float32 (0.00390625)
int16 ** int32 -> int32 (0)
int16 ** int64 -> int64 (0)
int16 ** int -> int64 (0)
int32 ** int8 -> float64 (0.00390625)
int32 ** int16 -> float64 (0.00390625)
int32 ** int32 -> float64 (0.00390625)
int32 ** int64 -> int64 (0)
int32 ** int -> int64 (0)
int64 ** int8 -> float64 (0.00390625)
int64 ** int16 -> float64 (0.00390625)
int64 ** int32 -> float64 (0.00390625)
int64 ** int64 -> float64 (0.00390625)
int64 ** int -> float64 (0.00390625)
int ** int8 -> int64 (0)
int ** int16 -> int64 (0)
int ** int32 -> int64 (0)
int ** int64 -> float64 (0.00390625)
int ** int -> float (0.00390625)
uint8 ** int8 -> int16 (0)
uint8 ** int16 -> int16 (0)
uint8 ** int32 -> int32 (0)
uint8 ** int64 -> int64 (0)
uint8 ** int -> int64 (0)
uint16 ** int8 -> int32 (0)
uint16 ** int16 -> int32 (0)
uint16 ** int32 -> int32 (0)
uint16 ** int64 -> int64 (0)
uint16 ** int -> int64 (0)
uint32 ** int8 -> int64 (0)
uint32 ** int16 -> int64 (0)
uint32 ** int32 -> int64 (0)
uint32 ** int64 -> int64 (0)
uint32 ** int -> int64 (0)
uint64 ** int8 -> float64 (0.00390625)
uint64 ** int16 -> float64 (0.00390625)
uint64 ** int32 -> float64 (0.00390625)
uint64 ** int64 -> float64 (0.00390625)
uint64 ** int -> float64 (0.00390625)
This is all extremely extremely weird and inconsistent...
Summary/edit by @seberg 2019-05-23:
On current master (and 1.16) we now error for many of these cases as found using the modified snippet:
signed_int_types = [np.int8, np.int16, np.int32, np.int64, int]
all_int_types = signed_int_types + [np.uint8, np.uint16, np.uint32, np.uint64]
for base_type in all_int_types:
for exponent_type in signed_int_types:
try:
result = base_type(4) ** exponent_type(-4)
except Exception as e:
result = e
print("{} ** {} -> {} ({})".format(base_type.__name__,
exponent_type.__name__,
result.__class__.__name__,
result))
Giving (again 1.16, master shortly before 1.17):
int8 ** int8 -> ValueError (Integers to negative integer powers are not allowed.)
int8 ** int16 -> ValueError (Integers to negative integer powers are not allowed.)
int8 ** int32 -> ValueError (Integers to negative integer powers are not allowed.)
int8 ** int64 -> ValueError (Integers to negative integer powers are not allowed.)
int8 ** int -> ValueError (Integers to negative integer powers are not allowed.)
int16 ** int8 -> ValueError (Integers to negative integer powers are not allowed.)
int16 ** int16 -> ValueError (Integers to negative integer powers are not allowed.)
int16 ** int32 -> ValueError (Integers to negative integer powers are not allowed.)
int16 ** int64 -> ValueError (Integers to negative integer powers are not allowed.)
int16 ** int -> ValueError (Integers to negative integer powers are not allowed.)
int32 ** int8 -> ValueError (Integers to negative integer powers are not allowed.)
int32 ** int16 -> ValueError (Integers to negative integer powers are not allowed.)
int32 ** int32 -> ValueError (Integers to negative integer powers are not allowed.)
int32 ** int64 -> ValueError (Integers to negative integer powers are not allowed.)
int32 ** int -> ValueError (Integers to negative integer powers are not allowed.)
int64 ** int8 -> ValueError (Integers to negative integer powers are not allowed.)
int64 ** int16 -> ValueError (Integers to negative integer powers are not allowed.)
int64 ** int32 -> ValueError (Integers to negative integer powers are not allowed.)
int64 ** int64 -> ValueError (Integers to negative integer powers are not allowed.)
int64 ** int -> ValueError (Integers to negative integer powers are not allowed.)
int ** int8 -> ValueError (Integers to negative integer powers are not allowed.)
int ** int16 -> ValueError (Integers to negative integer powers are not allowed.)
int ** int32 -> ValueError (Integers to negative integer powers are not allowed.)
int ** int64 -> ValueError (Integers to negative integer powers are not allowed.)
int ** int -> float (0.00390625)
uint8 ** int8 -> ValueError (Integers to negative integer powers are not allowed.)
uint8 ** int16 -> ValueError (Integers to negative integer powers are not allowed.)
uint8 ** int32 -> ValueError (Integers to negative integer powers are not allowed.)
uint8 ** int64 -> ValueError (Integers to negative integer powers are not allowed.)
uint8 ** int -> ValueError (Integers to negative integer powers are not allowed.)
uint16 ** int8 -> ValueError (Integers to negative integer powers are not allowed.)
uint16 ** int16 -> ValueError (Integers to negative integer powers are not allowed.)
uint16 ** int32 -> ValueError (Integers to negative integer powers are not allowed.)
uint16 ** int64 -> ValueError (Integers to negative integer powers are not allowed.)
uint16 ** int -> ValueError (Integers to negative integer powers are not allowed.)
uint32 ** int8 -> ValueError (Integers to negative integer powers are not allowed.)
uint32 ** int16 -> ValueError (Integers to negative integer powers are not allowed.)
uint32 ** int32 -> ValueError (Integers to negative integer powers are not allowed.)
uint32 ** int64 -> ValueError (Integers to negative integer powers are not allowed.)
uint32 ** int -> ValueError (Integers to negative integer powers are not allowed.)
uint64 ** int8 -> float64 (0.00390625)
uint64 ** int16 -> float64 (0.00390625)
uint64 ** int32 -> float64 (0.00390625)
uint64 ** int64 -> float64 (0.00390625)
uint64 ** int -> float64 (0.00390625)
So the integer+integer which use float as a common type are a bit strange in this sense.
Activity
[-]integer ** integer makes no sense for scalars[/-][+]NumPy's implementation of scalar integer ** integer has very weird casting[/+]charris commentedon Mar 22, 2016
Looks like array scalars are being used for some of the computations.
I also want to move this file into the
multiarraymodule at some point, starting with removing the immediate ufunc dependency and using the npymath library (which came later) instead. The file also needs other modernization.ewmoore commentedon Mar 23, 2016
The casting is only weird for negative exponents. For positive it's the same as the other binary operators.
What's happening is that the scalar binary operator routines fall back to just calling the equivalent ufunc when both types cannot be cast to the type of the first. For
**, this give the strange casting table because the scalar operation tries to look at the values to determine the output type, but the ufunc doesn't. The fix is probably to try and implement the effective__rop__logic in the scalar operation without just giving up and calling the ufunc unless one of the arguments is not a scalar type.If we don't want to return a float for integer-integer powers, then we can just stop looking at the values to determine the output type.
This quick fallback to calling a ufunc also mean that mixed type scalar operations can be roughly 10x slower (here at least) than the pure type scalar operation depending of the order of the types:
ENH/BUG: try hard not to fall back to a ufunc call for scalar ops
ENH/BUG: try hard not to fall back to a ufunc call for scalar ops
charris commentedon Nov 17, 2018
Pushing this issue off to 1.17. I don't know the current status of these problems, or whether #8126 is still needed. We might want to implement #8126 differently. @mattip Ping because you have been working in that area for matmul.
mhvk commentedon Jun 12, 2019
Removing the milestone - I think issues should have them only if release-critical...