You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Stop mixing up integer and floating point overflow #2898
The integer arithmetic code in scalarmathmodule.c.src has overflow-detection logic in it. That's great. When it detects an overflow, the way it signals that is by calling npy_set_floatstatus_overflow(). That's not so great.
Two problems:
Integer rollover is really not the same thing as floating point overflow. (I'm much more worried about integer overflow for one thing, since it doesn't nicely go to inf, it just scrambles your data in some impossible to notice way.)
Setting and checking the float status bits is REALLY EXPENSIVE. Right now we set and check the float status bits on every scalar integer operation, and this makes those operations slower by a factor of two. So yeah, we shouldn't do that.
Proposed solution:
Add a new arithmetic error condition, let's call it "rollover", which is specifically for integer overflow, and make the current "overflow" just be used for floats. (So this is a new argument to np.seterr, etc.)
Test and signal this condition using a new thread-local storage key, rather than the floating point status bits. (This should be much cheaper, though you know, double check this.)
Python exports an undocumented C API for thread-local storage:
/* Initialize a thread-local storage spot for our flag during module setup */
static int integer_error_flag_key = PyThread_create_key()
/* Clear the flag before running integer code */
/* NB this sets its value to NULL, and set_key_value is a no-op if this is not called first */
PyThread_delete_key_value(integer_error_flag_key);
/* Raise the flag when necessary */
PyThread_set_key_value(integer_error_flag_key, (void *) 1);
/* Check the flag */
if (PyThread_get_key_value(integer_error_flag_key) != NULL) {
/* aieee */
}
Closing, since the code in question no longer has integer overflow detection at all. The idea to set a thread-local-storage flag can be revived when we revisit the overflow problem with a new dtype. Please reopen if I misunderstood.
Activity
mattip commentedon Sep 14, 2019
Closing, since the code in question no longer has integer overflow detection at all. The idea to set a thread-local-storage flag can be revived when we revisit the overflow problem with a new dtype. Please reopen if I misunderstood.