Repository navigation
BUG: integer promoted to float during calculation #23476
Description
Describe the issue:
I have an array of integers representing the index space of a Cartesian mesh. From this I calculate the dimensions of the mesh. The calculation should be integer operations. Both the operands and result should be integers.
Reproduce the code example:
import numpy as np
ext = np.array([0,3,0,4], dtype=np.uint64) # an array representing the index
# space of a Cartesian mesh indexed by i and j
# in the format [i0, i1, j0, j1].
nx = ext[1] - ext[0] + 1 # the mesh dimension in the i direction. this should
# be an integer but numpy unexpectedly promotes to
# float64 during the calculation.
print('type of ext[0] = %s'%(str(type(ext[0]))))
print('type of nx = %s'%(str(type(nx))))Error message:
no imediate crash or errors, however, promoting to float during operations on integers is unexpected behavior and causes issues in later calculations which should be integer operations and relay on the behavior of integer operations for correctness.
Runtime information:
1.24.2
3.11.2 (main, Feb 8 2023, 00:00:00) [GCC 12.2.1 20221121 (Red Hat 12.2.1-4)]
[{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C'],
'not_found': ['FMA3',
'AVX2',
'AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_KNM',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'architecture': 'Sandybridge',
'filepath': '/home/bloring/work/teca/py3k-mpich-cuda/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-15028c96.3.21.so',
'internal_api': 'openblas',
'num_threads': 20,
'prefix': 'libopenblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.21'},
{'filepath': '/usr/lib64/libgomp.so.1.0.0',
'internal_api': 'openmp',
'num_threads': 20,
'prefix': 'libgomp',
'user_api': 'openmp',
'version': None}]
None
Context for the issue:
The unexpected promotion from int to float leads to incorrect results in later calculations that depend on the standard behavior of integer operations. It also results in issues when interoperate with C++ codes that expect integers.
If I instead use a Python list, rather than a numpy array the behavior is as expected, integer arithmetic operations are used and the result is an integer.
ext = [0,3,0,4] # and array representing the index
# space of a Cartesian mesh indexed by i and j
# in the format [i0, i1, j0, j1].
nx = ext[1] - ext[0] + 1 # the mesh dimension in the i direction.
# Python does not promote this to a float.
# This is the correct and expected behavior.
print('type of ext[0] = %s'%(str(type(ext[0]))))
print('type of nx = %s'%(str(type(nx))))
Activity
seberg commentedon Mar 28, 2023
There are two possible angles to this:
NPY_PROMOTION_STATE=weakwould test ride such changes, but there are still some loose ends see NEP 50 and discussion welcome :). E.g. should this apply mainly to operators, how do we define "cast safety" for a python integer?).int64anduint64(a bit more generic) promote tofloat64, which is pretty weird and BUG: Binary operations between uint64 and intX results in float64 #20905 discusses it for example.The second one is maybe more to the point (I guess this is a duplicate of it). Unfortunately, it's complicated (mostly w.r.t. to backcompat), but there is some discussion over there.
burlen commentedon Mar 28, 2023
OK, I see a bunch of others have suffered from this.
The crux of this issue is that promotion from integer to floating point type during arithmetic should only occur if one of the operands are floating point. This is essential behavior since the difference between integer and floating point arithmetic is useful and we'd like to exploit that in our codes. Promoting to floating point when all operands are integers prevents the exploitation of this behavior. At the very least it's confusing and unexpected.
Why wouldn't numpy implement the behavior defined by C/C++ relating to these issues? (ref)
seberg commentedon Mar 29, 2023
I don't think you should care about the why here (yes, Chestertons fence, but overall...). These things presumably pre-date NumPy. There is at least one more reason: The fact that until recently numpy used to clump together the concept of promotion and "safe casting" (which is probably the argument for the original promotion behavior).
The problem is not about convincing anyone that the status-quo is not a great definition. It is about having a plan of how to enact change and to what extend (compared to e.g. C rules which are quite broadly different!).
seberg commentedon Feb 3, 2024
Closing, the examples are fixed in main/NumPy 2.0 by NEP 50. For the mixing of uint/int see the issue linkes above (gh-20905).