BUG: timedelta64 comparison is not transitive

Describe the issue:

The logic for timedelta64 comparisons involving a generic timedelta64 is surprising, and leads to non-transitivity of something that I'd expect to be a total ordering. In the snippet below, x < y and y < z are both True, but x < z is False.

>>> import numpy as np
>>> x, y, z = np.timedelta64(1, "ms"), np.timedelta64(2), np.timedelta64(5, "ns")
>>> x < y < z
np.True_
>>> x < z
np.False_

The x < z result is what I expect. For x < y and y < z, I was expecting either a consistent default for the units for y (though I'm not sure what default I was expecting - any of days / seconds / nanoseconds would seem plausible), or to get an error or warning about an ill-defined comparison.

It appears that the actual behaviour is that y is interpreted as being in milliseconds for the x < y comparison, and then in nanoseconds for the y < z comparison.

The same lack of transitivity applies to equality comparisons:

>>> x, y, z = np.timedelta64(1, "ms"), np.timedelta64(1), np.timedelta64(1, "ns")
>>> x == y == z
np.True_
>>> x == z
np.False_

(Though it looks like we're protected from peculiar dict / set containment consequences by the fact that y isn't hashable.)

Meta: "BUG" may be a stretch here - I don't doubt that the behaviour described is intentional, but it's still surprising, and as far as I can tell undocumented (I couldn't find any hints to this behaviour at https://numpy.org/doc/stable/reference/arrays.datetime.html). In which case this is at best a doc bug.

Reproduce the code example:

import numpy as np
x, y, z = np.timedelta64(1, "ms"), np.timedelta64(2), np.timedelta64(5, "ns")
print("x < y < z:", x < y < z)
print("x < z:", x < z)

Error message:

Python and NumPy Versions:

NumPy 2.2.2
Python 3.13.1 (main, Dec 7 2024, 10:29:14) [Clang 16.0.0 (clang-1600.0.26.4)]

Runtime Environment:

No response

Context for the issue:

This doesn't affect my work - we've already worked around this issue in our own code. (I think the lesson learned is "to avoid surprises, always provide a unit when using np.timedelta64".)

#14622 seems tangentially related - it looks as though this is where the non-hashability of the generic timedelta64 was introduced.

or to get an error or warning about an ill-defined comparison

The same problem exists with np.timedelta64(3, "s") == 3. I would agree with changing that behavior, although it has come up before that for this change we may have to go to an error.

I think once we decide to do something about that case, it could make sense to work on (slowly or not) effecitvely getting rid of naive timedelta.

(There is a reason that things like datetime(..., "s") + 1000 magically work via naive timedelta. But we do a better job about it now, and over-all I don't think that magic is even relied on in practice.)

Maybe the solution is even to go to an error but allow np.errstate(...) to change it (i.e. to disable the error and have np.timedelta64(3, "s") == 3 return False).

Hello, I would like to work on this issue
So, the I guess the issue comes from this function: get_datetime_conversion_factor.
How we would like to proceed ? raising and error and not allow this ? or raising a warning and keep the actual behavior ?

Hello! I'm a first comer to this repository and I was wondering if I could work on this issue? It is my first time ever contributing to an open-source program and I believe it could be a good way to test and improve my programming skills.

This is probably not a good first issue.

I would like to try to solve it anyway, if that's okay

You can work on whatever you like, but don't be surprised if this is a very deep change (i.e. in code but also in what you need to understand) and also one that requires more discussion than programming to make progress.

Thank you for your answer. I will be working on another issue then.

After investigating the source code, I propose raising a RuntimeWarning when casting from a unitless np.timedelta64 (generic) to a unitful np.timedelta64 as a temporary solution. This is intended to encourage users to pay attention to the behavior.

Details

Raise RuntimeWarning at unit conversion function (below)

numpy/numpy/_core/src/multiarray/datetime.c

Lines 1037 to 1042 in 8b89d0f

    
           /* Generic units change to the destination with no conversion factor */ 
        
           if (src_meta->base == NPY_FR_GENERIC) { 
        
               *out_num = 1; 
        
               *out_denom = 1; 
        
               return; 
        
           }

Disable view_offset when the unit is generic.

Rationale

If a np.timedelta64 object has a generic unit, the unit conversion function will not be called because view_offset is set to True.

numpy/numpy/_core/src/multiarray/datetime.c

Lines 3889 to 3893 in 8b89d0f

    
           else if (meta1->base == NPY_FR_GENERIC) { 
        
               if (byteorder_may_allow_view) { 
        
                   *view_offset = 0; 
        
               } 
        
               return NPY_SAFE_CASTING ;

numpy/numpy/_core/src/umath/ufunc_object.c

Lines 787 to 790 in 8b89d0f

    
           if (view_offset != 0) { 
        
               /* NOTE: Could possibly implement non-zero view offsets */ 
        
               must_copy = 1; 
        
           }

If it makes sense, I'll submit a PR for that.

This issue is about 6 months old. If it would be more appropriate to open a new issue for this proposal, please let me know and I’ll be happy to do so.

ENH: raise RuntimeWarning when casting from unitless timedelta to uni…

[ENH] Raise RuntimeWarning when casting from a unitless `np.timedelta…

ENH: Raise RuntimeWarning when casting from a unitless `np.timedelta6…

@seberg

@seberg

I think it would be ideal to encourage users to define "NaT" with a specific unit, such as np.timedelta64("NaT", "ns").
But, if this turns out to be too inconvenient in practice and we have to address it, I have come up with two options:

1. Keep generic unit internally as a private unit and only allow "NaT" to be created with the generic unit.
1. Assign a specific unit to "NaT" when it is created without unit. (e.g., assign "ns" unit to it)

Personally, I prefer completly removing the generic unit. ~~but I think option 2 is a reasonable compromise in terms of codebase simplicity. What do you think?~~

On second thought, since np.datetime64 currently allows unitless objects only for NaT,
timedelta64 should follow the same approach. That corresponds to the first option, which seems the most straightforward.

ENH: Raise RuntimeWarning when casting from a unitless `np.timedelta6…