-
-
Notifications
You must be signed in to change notification settings - Fork 11.8k
ENH, FEAT: Reorganize finfo and add new constant slot #29836
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Unfortunately, I also need getitem nicer, so tried to change that. There is currently some somewhat larger fallout (and voids fail) Signed-off-by: Sebastian Berg <[email protected]>
seberg
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a few small comments. It seems like at least one run fails around querying the subnormal information from the macros.
Not sure what to do about that, I suppose either hard-code or maybe use nextafter for them.
(if we do that, it might be nice to see if we can find out whether subnormals work in that step, no idea what nextafter does. OTOH, the typical thing is FTZ mode, which I suspect still allows manual creation with nextafter... so it's really a user problem and we would need an ftz attribute on finfo rather than changing what smallest_subnormal gives.)
| return NULL; | ||
| } | ||
| PyObject *finfo = PyTuple_GetItem(args, 0); | ||
| if (finfo == NULL || finfo == Py_None) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't be NULL, but doesn't matter. I might not bother, it'll just say "can't set attributes of NoneType" without this, but happy either way.
| #define NPY_CONSTANT_finfo_nmant 13 | ||
| #define NPY_CONSTANT_finfo_min_exp 14 | ||
| #define NPY_CONSTANT_finfo_max_exp 15 | ||
| #define NPY_CONSTANT_finfo_decimal_digits 16 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be nice to see opinions about the choice of constant here, and also the constants derived in Python.
I tried modifying the template to use nextafter when macro is not defined, only hardocded the16 bit (it was going 0) edit: wait, maybe I can do better, let me fix all the other error cases first |
|
Any reason why for double-double format tiny was set to NAN instead of |
IIRC it was some weird/wrong value, then things were changed and nobody wanted to figure out the right value, so it was set to NaN because that seemed still better than being nonsense. |
I think Let me confirm this and post here whatever suits the best |
|
Only override the |
|
For this particular issue of testing array_repr numpy/numpy/_core/tests/test_longdouble.py Lines 288 to 294 in 50204c8
platforms where numpy/numpy/_core/printoptions.py Line 18 in 50204c8
Windows Old behaviour (before refactoring)Resulting in 2 arrays as following In [1]: import numpy as np
In [2]: np.__version__
Out[2]: '2.3.3'
In [3]: LD_INFO = np.finfo(np.longdouble)
In [4]: o = 1 + LD_INFO.eps
In [5]: a = np.array([o])
In [6]: b = np.array([1], dtype=np.longdouble)
In [7]: a, b
Out[7]: (array([1.]), array([1.], dtype=float64))As you can see the only difference is in dtype leading to that test's assert to fail leading to passing the test In this new behaviourIn [1]: import numpy as np
In [2]: np.__version__
Out[2]: '2.4.0.dev0+git20250930.d8508a2'
In [3]: LD_INFO = np.finfo(np.longdouble)
In [4]: o = 1 + LD_INFO.eps
In [5]: a = np.array([o])
In [6]: b = np.array([1], dtype=np.longdouble)
In [7]: a, b
Out[7]: (array([1.], dtype=float64), array([1.], dtype=float64))So since both get dispatched from the same dtype, results in failing the test, I am not sure whether testing this was the intention behind this test. Anyways the hack to retain the original behaviour is as follows inside # On platforms where longdouble is the same size as double (e.g., Windows),
# use double descriptor to populate constants for backward compatibility.
# The old MachArLike code would match the float64 signature on such platforms
# and return float64 scalars.
if (self.dtype.type == ntypes.longdouble and
self.dtype.itemsize == numeric.dtype(ntypes.double).itemsize):
self.dtype = populate_dtype
populate_dtype = numeric.dtype(ntypes.double)
else:
populate_dtype = self.dtype
# Fills in all constants defined directly on the dtype (in C)
_populate_finfo_constants(self, populate_dtype)This override to use numpy/numpy/_core/arrayprint.py Lines 1520 to 1562 in 50204c8
@seberg what you suggest? keeping the current way (which retain the past behaviour) or need to change the array print behaviour by increasing the precision to respectively used dtype and ignorning the name of dtype (Not sure how useful, maybe can do in a different PR)? |
|
The Qemu/loongarch64 docker image is not even building so considering that out for now. This PR is ready for the review |
seberg
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just leaving as comments. I like this, how happy are you with the approach?
Did you have a look at the derived values I had already created? The other question is whether we should rescue some of the old parts (much reduced) and slam them into tests?
Need to have another pass, but wanted to give these comments. But also, I think it's basically done.
(With this change, I assume there may be weird platform fallouts eventually, but that is something to deal with when it happens.)
Co-authored-by: Sebastian Berg <[email protected]>
| #define NPY_CONSTANT_maximum_finite 4 | ||
| #define NPY_CONSTANT_minimum_finite 5 | ||
| #define NPY_CONSTANT_inf 6 | ||
| #define NPY_CONSTANT_ninf 7 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this use a clearer name, e.g. neg_inf
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good, I know I added it, but I guess we could even remove it and define it in Python as -inf. I think we removed np.ninf after all. (But I don't mind either way).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might as well still change this I guess.
Co-authored-by: Sebastian Berg <[email protected]>
Co-authored-by: Sebastian Berg <[email protected]>
Co-authored-by: Sebastian Berg <[email protected]>
|
@seberg I need an opinion here From macros FLT_MANT_DIG=24 includes the implicit leading bit for precision (IEEE specifies 23 explicit fraction bits but effective 24-bit significand); So do you prefer the complete original behaviour or we should take the whatever value comes from the header as finfo and adjust the derived values correct with proper documentations around this? |
|
Also orignally machar used values make more sense atleast for |
Ufff, that is annoying that these definitions are so subtly different! For the Python side, we clearly can't change the values. For (I would like to still tag the integer constants, but if you like I can push that also.) |
| /* | ||
| Definition: Minimum negative integer such that FLT_RADIX raised by power one less than that integer is a normalized float, double and long double respectively | ||
| refernce: https://en.cppreference.com/w/c/types/limits.html |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left these comments for developers so that they can reference why we are subtracting 1
| from numpy import exp2, log10 | ||
|
|
||
|
|
||
| class MachArLike: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file mimics the old machar behaviour to test
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
kept in separate file since machar is not going to be around so not mixing this with other tests
SwayamInSync
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes ensure the values remains as original and we can document them in new finfo docs on what C macros returns and where we tweak it for backwards compatibility.
This way if user will be aware of the behaviour and if needs the true C macro value then he can just add +1 to the respective fields to get them.
|
@seberg if this look good enough then I can proceed with the documentation |
|
Hmm... test failures aren't looking to be coming from finfo refactor |
|
@SwayamInSync decided to just put it in. I feel this should be reasonable both from a general API perspective and from the changes to the (Although, I will expect that there may be some follow-ups with finfo, it's always trickier than you expect.) I think it's just as well to include release notes that you think are helpful in a seperate PR to add docs (and at that point -- or earlier -- others will also get another chance to review the API choices here). |
|
Thanks @seberg , will do the doc PR asap |
Introduce `NPY_DT_get_constant` as a slot and a corresponding function that fills in a single element pointer for an arbitrary (corresponding) dtype/descriptor. Since we do want some integer values for `finfo`, some values are set to always fill in an `npy_intp` value instead. The slot assumes that it will be used with the GIL held and on uninitialized data, since I suspect that this is the typical use-case (i.e. it is unlikely that we need to fetch a constant deep during a calculation, especially in a context where the correct type/value isn't known anyway). This completely re-organizes and simplifies `finfo` by moving all definitions to C. Co-authored-by: Sebastian Berg <[email protected]>
TYP: update the ``finfo`` stubs
Introduce `NPY_DT_get_constant` as a slot and a corresponding function that fills in a single element pointer for an arbitrary (corresponding) dtype/descriptor. Since we do want some integer values for `finfo`, some values are set to always fill in an `npy_intp` value instead. The slot assumes that it will be used with the GIL held and on uninitialized data, since I suspect that this is the typical use-case (i.e. it is unlikely that we need to fetch a constant deep during a calculation, especially in a context where the correct type/value isn't known anyway). This completely re-organizes and simplifies `finfo` by moving all definitions to C. Co-authored-by: Sebastian Berg <[email protected]>
…umpy#29836 (numpy#29889) This is a follow-up PR of work happened in numpygh-29836 * The deprecated MachAr runtime discovery mechanism has been removed. * np.finfo fetches the constants provided by the compiler macros * new slot to fetch the dtype related constants
closes #27231
Summary
This pull request removes the legacy
MachArclass and related files from NumPy, streamlining the codebase and updating the internal dtype API to support querying numerical constants directly from dtypes. The changes modernize how machine limits and constants are handled, moving away from Python-side introspection to a more robust and extensible C API.Removal of legacy Python code:
numpy/_core/_machar.pyfile, which contained the deprecatedMachArclass for floating-point machine parameter introspection.numpy/_core/_machar.pyiand its references from the build configuration, fully eliminating Python-side support forMachAr.numpy/_core/__init__.pyto remove_machar, reflecting the removal of the module.numpy/__init__.pyas they are no longer needed with the new approach.Enhancements to dtype API for numerical constants:
NPY_CONSTANT_zero,NPY_CONSTANT_one,NPY_CONSTANT_maximum_finite, etc.) and a newPyArrayDTypeMeta_GetConstantfunction to the dtype API, enabling direct querying of numerical constants from dtypes in C.numpy/_core/include/numpy/dtype_api.hto accommodate the new constants and API changes.Miscellaneous codebase improvements:
#include <float.h>innumpy/_core/src/multiarray/arraytypes.c.srcto ensure access to floating-point limits in C.numpy/dtype_api.hheader inarraytypes.c.srcto support the new constant querying API.