I am working on a talk I would like to give at PyOhio about float(‘nan’) and I was wondering if anyone knows why each call to float(‘nan’) returns a different instance? TIA.
It’s a constructor, same as calling list() will give a different instance every time. If you specifically want the SAME instance, use math.nan instead - since that’s an attribute lookup, it’ll be the same object each time.
1 Like
So the reason that each one is different is because it’s a constructor? Again, I’m looking for some historical background about why each one is different for a talk, not for actual usage.
Floats aren’t interned. You can construct multiple objects with the same value:
>>> x = float("1234.567")
>>> y = float("1234.567")
>>> x is y
False
Normally, when you construct a new object by calling the class, you get back a new and unique object. SOME objects special-case this and will return the same object (eg when you call bool, you will always get back one of the two predefined booleans, and if you call type(None)() you will get back None), but the normal behaviour of most types is that you get distinct objects.
WHY are they different? Mostly because it’s extra effort to check if one already exists. That effort is worthwhile in some cases (for example, most versions of Python have a preconstructed set of small integer objects - though the exact range of that varies by version and implementation), but otherwise, it won’t be done. If you want to guarantee that the exact same object will be used every time, that’s called interning; sys.intern(some_string) will return that same string, but always using the same object for the same value. You can do the same for floats using a dictionary:
_ALL_FLOATS = {}
def float_intern(f):
return _ALL_FLOATS.setdefault(f, f)
If I repeat the exercise from earlier, but calling this function:
>>> x = float_intern(float("1234.567"))
>>> y = float_intern(float("1234.567"))
>>> x is y
True
we find the same object each time. This can save memory (if the objects are large, which is why this can be done with strings - but the dictionary itself consumes space too), and it can speed up equality comparisons, but neither of those is likely to be very significant for floats.
So, yes, the reason they’re different is because it’s a constructor, and by default, that’s what constructors do.
7 Likes
One of the important properties of NaN, sometimes the test to see if a float is a NaN, is that NaN != NaN. If float('nan') and float('nan') were the same instance, and if float.__eq__ was not overridden then Python’s default behaviour for == would mean NaN == NaN, contradicting IEEE 754.
>>> x = float('nan')
>>> object.__eq__(x,x)
True
>>> float.__eq__(x,x)
False
There are quiet NaNs, loud NaNs and all sorts of free bits within the binary implementation of floats though, so however any particular programming language chooses to support them is its own design choice.
4 Likes
Because all types are (direct or indirect) subtypes of
object, they inherit the default comparison behavior fromobject. Types can customize their comparison behavior by implementing rich comparison methods like__lt__(), described in Basic customization.The default behavior for equality comparison (
==and!=) is based on the identity of the objects. Hence, equality comparison of instances with the same identity results in equality
…
The not-a-number values
float('NaN')anddecimal.Decimal('NaN')are special. Any ordered comparison of a number to a not-a-number value is false. A counter-intuitive implication is that not-a-number values are not equal to themselves. For example, ifx = float('NaN'),3 < x,x < 3andx == xare all false, whilex != xis true. This behavior is compliant with IEEE 754.
1 Like
Not for nan’s, of course.
Though, CPython set internally just one, provided by NAN macro. It’s a quiet NaN, but nothing else is specified by the C standard (sign bit or payload).
But you could set a different nans and it’s properties will be respected by floating-point arithmetic on IEEE boxes. E.g. you can change sign bit:
>>> from math import nan, copysign, isnan
>>> copysign(1.0, nan)
1.0
>>> neg_nan = -nan
>>> copysign(1.0, neg_nan)
-1.0
>>> isnan(neg_nan)
True
Or you can add payload to nan, using the struct module:
>>> import struct
>>> def make_nan(sign, quiet, payload):
... payload_mask = 0x7ffffffffffff
... i = (sign << 63) + (0x7ff << 52) + (quiet << 51)
... return bytes.fromhex(f"{i + payload:x}")
...
>>> make_nan(1, 0, 123)
b'\xff\xf0\x00\x00\x00\x00\x00{'
>>> struct.unpack('>d', _)[0]
nan
That payload will be propagated in arithmetic operations according to the standard on IEEE boxes.
So, technically, you can have different (internally) NaNs.
1 Like
No, but we do for anything that’s equal. Since “Not a number” is a category rather than a single value, you can have more than one of them even after interning. But for everything else - including infinities and subnormals - you’ll get the same object back.
The language doesn’t define this. It’s an accident of the implementation, and does vary across implementations. Under CPyhon 3.14.1:
>>> for x in '-12.7', '-inf', 'nan':
... print(x, float(x) is float(x))
...
-12.7 False
-inf False
nan False
Same thing under PyPy 7.3.12:
>>>> for x in '-12.7', '-inf', 'nan':
.... print(x, float(x) is float(x))
-12.7 True
-inf True
nan True
Nobody cares; and tracing down implementation details is boring 
There is no Grand Plan™ or something interesting going on - just whatever fell out of the implementation choices made.
5 Likes
Exactly.
Though, I would note, that such “caching” seldom useful for floats (c.f. ints, for instance).
Indeed. The language define default behavior for equality comparison:
The default behavior for equality comparison (
==and!=) is based on the identity of the objects. Hence, equality comparison of instances with the same identity results in equality, and equality comparison of instances with different identities results in inequality.
But then float(‘nan’) and Decimal(‘nan’) noted as special (snowflakes) in this regard. So, implication “nans are unequal” => “they should be different objects wrt to id()” — will be wrong.
NaN could be like None. This harm compatibility with the IEEE 754, however.
Yeah, it’s a poor tradeoff, but a valid demonstration.
FYI, Python floating point numbers (double precision floats) have over 9 quadrillion (2 * 2**52) distinct NaN values. With current RAM prices, I would not cache NaNs.
5 Likes
Nevertheless, under the covers CPython doesn’t work with raw machine bits. Everything is a “Python object”, which is a C struct that holds at least:
- A representation of the raw machine bits.
- A pointer to the object’s type object (itself also a “Python object”).
- The current reference count.
The internal string-to-float routines return a C double, which is then packaged inside a Python float object. It’s the addresses of these objects (C structs) that determine the outcome of “is”, and they’re never equal regardless of which machine doubles they wrap.
It’s the same story for every type, but some immutable types go on to make special cases of “very frequent” values. Like very small integers. The details don’t matter, because nothing about this is guaranteed. No such special-casing happens to be done by CPython for floats (regardless of their values).
Nevertheless, you can get the same id (memory address in CPython) for different floats, but only provided they’re not alive simultaneously. Like so:
>>> id(float('nan'))
2801032084496
>>> id(float("-3.14"))
2801032084496
That simply means that CPython happened to reuse the very same memory allocated. That’s fine because reference counting destroyed the object created for “nan” when id() finished wrapping its address as a Pythion int object, so there’s no reason not to reuse the memory immediately after.
But nothing about that is guaranteed either. CPython list objects happen to save away the memory addresses of some number of dead float objects, to save the full expense of “start over from scratch” memory allocation and deallocation as floats pop in and out of existence.
PyPy works very differently, and that’s fine too. I don’t know its details - and don’t care In many cases, PyPy does work with the raw machine bytes directly, which can save tons of RAM and cycles.
2 Likes
Another step down the rabbit hole:
>>> a = 1.0
>>> b = 1.0
>>> id(a) == id(b)
False
Not defined by the language. The float-to-string is done at compile-time instead of run-time now, but same result: two distinct Python objects are created, and the float constructor does nothing to try to see whether they refer to the same C double.
So what’s up with this?
>>> if 1:
... a = 1.0
... b = 1.0
... id(a) == id(b)
True
Again undefined, but the result is different! Nothing actually changed: the compiler still found two distinct float objects, but compilation is done “a code block” as a whole. Part of the code object constructed is a table of constants encountered, and it’s that part - unique to compilation - that searches the constants found so far for a match to a new constant. That’s the part that recognizes the second 1.0 is redundant, so throws its object away and refers instead to the original occurrence.
Move to compiling a new code block, and all previous constants found are forgotten. Each code block has its own code object, and so also its own table of constants.
All imlementation consequences, none defined, and none that should be relied on.
5 Likes
As I said above, usually we will have only two different (sign bit) NaNs. In principle, some Python implementation could ignore and that 
Python docs don’t say somewhere, that implementation must follow to the IEEE 754 standard. (We have no way even to check conformance to this standard like sys.ieee754 is True.)
So, wrt to bit content, there could be just one NaN, like in the mpmath:
>>> a = mpf('nan')
>>> b = mpf('nan')
>>> a._mpf_
(0, mpz(0), -123, -1)
>>> b._mpf_
(0, mpz(0), -123, -1)
>>> a._mpf_ is b._mpf_
True
>>> (-a)._mpf_
(0, mpz(0), -123, -1)
Though, your argument is valid for finite floats. But size of set is not a reason against caching: set of integers is infinite. Well, in practice it’s restricted by maximal bit count for CPython integers, which is around exbibyte. Anyway, much worse than for floats, not taking into account size of distinct integers.
But floats have no useful subsets like small integers to cache.
1 Like
No, NaNs typically ignore the sign bit. On the other hand, some bits of the mantissa have special meaning. They indicate if it’s a signaling NaN or a quiet NaN. On modern LE arch it’s often the first bit. Sometimes the rest of the payload is used for special purposes, too.
Not, if we are talking about arithmetic. Above was an example, though it’s better with new signbit function:
>>> from math import nan, signbit
>>> signbit(nan)
False
>>> signbit(-nan)
True
Same about payload:
>>> def make_nan(sign, quiet, payload):
... import struct
... payload_mask = 0x7ffffffffffff
... i = (sign << 63) + (0x7ff << 52) + (quiet << 51)
... return struct.unpack('>d', bytes.fromhex(f"{i + payload:x}"))[0]
... def get_payload(nan):
... import struct
... b = struct.pack('>d', nan)
... n = int.from_bytes(b)
... return n & 0x7ffffffffffff
...
>>> nan1 = make_nan(1, 0, 123)
>>> nan2 = make_nan(0, 0, 321)
>>> get_payload(nan1 + 1)
123
>>> get_payload(1 + nan1)
123
>>> get_payload(nan2 + 1)
321
>>> get_payload(nan1 + nan2) # this is not defined by IEEE 754, may be 321
123
>>> import math
>>> get_payload(math.atan2(nan1, nan2)) # atan2 like second one
321
Though, it’s sometimes ignored in CPython, notably in the cmath module:
>>> import cmath
>>> get_payload(math.sin(nan1)) # libm works
123
>>> get_payload(cmath.sin(nan1).real) # cmath smash payload
0
>>> import ctypes
>>> libm = ctypes.CDLL('libm.so.6')
>>> libm.csin.argtypes = [ctypes.c_double_complex]
>>> libm.csin.restype = ctypes.c_double_complex
>>> get_payload(libm.csin(nan1).real) # libm rulez
123
Yes, though I don’t remember when seen this in the real world.
But what I meant there is no simple interfaces to get/set all this stuff from Python world (struct support for this was added recently) and that is why on practice we have just two distinct nan’s.
2 Likes
I just want to take a moment to thank everyone who responded here. While it looks like the answer to my original question might not be something I can find out, I did learn a lot of things that will be useful to me in preparing my talk.
2 Likes
It was the intent of the standard that a NaN payload could be used to record retrospective debugging aids (e.g., record the address of the instruction that produced a NaN), but I don’t recall anyone ever implementing that intent. There are some non-intended uses, though. For example, I’m told that the R statistical package reserves one specific NaN bit pattern for “missing data”, to distinguish that case from “numerical error”. To my eyes, that’s a good, practical use.
As I recall, it’s not defined at all by the standard. That an operation reproduce the payload of an input NaN (if any) is just recommended (“should”), not required (“shall”).,
3 Likes
I don’t think they are “not-intended”. Standard mention other possible uses, e.g.:
Signaling NaNs afford representations for uninitialized variables and arithmetic-like enhancements (such as complex-affine infinities or extremely wide range)
Quiet NaNs should, by means left to the implementer’s discretion, afford retrospective diagnostic information inherited from invalid or unavailable data and results.
You are right. It’s not required, yes. But defined by the standard as an important optional feature (“should”, not “may”).
The problem is that only recent editions of the IEEE 754 define where payload “should” be (i.e. to distinguish it’s bits from encoding of the NaN type). That partially explains poor adoption of this IEEE feature.
1 Like