DOC: Inconsistencies in converting array with datetime values into object array

When converting a datetime64[ns] array into object type, it returns the date as an integer instead of datetime.datetime object

Reproducing code example:

>>> import numpy as np
>>> from datetime import datetime
>>> d = datetime.now()
>>> np.array(d)
array(datetime.datetime(2021, 8, 29, 20, 1, 32, 350877), dtype=object)
>>> np.array(d, dtype="datetime64[ns]")
array('2021-08-29T20:01:32.350877000', dtype='datetime64[ns]')
>>> np.array(d).astype("O") # Keeps the same, as expected
array(datetime.datetime(2021, 8, 29, 20, 1, 32, 350877), dtype=object)
>>> np.array(d, dtype="datetime64[ns]").astype("O") # Transforms to int. Unexpected
array(1630267292350877000, dtype=object)
>>> np.array(d, dtype="datetime64[ns]").view(np.int64) # Transforms to int. Expected
array(1630267292350877000)

IMO, the results from np.array(d).astype("O") and np.array(d, dtype="datetime64[ns]").astype("O") should be equal

NumPy/Python version information:

1.18.5 3.8.3 (default, Jul  2 2020, 11:26:31) 
[Clang 10.0.0 ]

There is no inconsistent Found , all works good .

For reference: the scalar type returned by datetime64.item() (or astype("O") in your case) depends on the unit, though I'm not sure if this actually documented somewhere?

In [1]: import numpy as np

In [2]: unit_list = ["Y", "M", "D", "h", "m", "s", "ms", "us", "ns", "ps", "fs", "as"]

In [3]: for unit in unit_list:
   ...:     scalar = np.datetime64(0, unit).item()
   ...:     print(f"datetime64[{unit}]: {type(scalar)}")
   ...: 
datetime64[Y]: <class 'datetime.date'>
datetime64[M]: <class 'datetime.date'>
datetime64[D]: <class 'datetime.date'>
datetime64[h]: <class 'datetime.datetime'>
datetime64[m]: <class 'datetime.datetime'>
datetime64[s]: <class 'datetime.datetime'>
datetime64[ms]: <class 'datetime.datetime'>
datetime64[us]: <class 'datetime.datetime'>
datetime64[ns]: <class 'int'>
datetime64[ps]: <class 'int'>
datetime64[fs]: <class 'int'>
datetime64[as]: <class 'int'>

@BvB93

Hi @BvB93 , thanks for answering.

Is there a reason behind this behavior? Where can I find it?

This is annoying especially when using pd.Series, since when calling values in a series it returns by default an array with type datetime64[ns]:

>>> import pandas as pd
>>> from datetime import datetime
>>> pd.Series(datetime.now()).values.item()
1630334959380213000

Is this desired/expected?

Is there a reason behind this behavior? Where can I find it?

I'm afraid this was a bit before my time; I'm not familiar with the exact rationale here (though admitetly I'm not a huge fan of the practice).

This is annoying especially when using pd.Series, since when calling values in a series it returns by default an array with type datetime64[ns]:

This does sounds like more of a pandas problem though. Numpy itself won't automatically construct a datetime64 array from a datetime.datetime instance, instead returning an object array. The decision to automatically create a datetime64[ns] array is one that is made by pandas.

In [1]: import numpy as np
   ...: import datetime as dt

In [2]: np.array(dt.datetime.now()).dtype
Out[2]: dtype('O')

@BvB93

I agree, I'm gonna left the issue open because I would like to see the docs/explanation on the first behavior that's not intuitive.

Besides that, everything else makes sense. Thank you @BvB93 !

I believe Pandas settled on np.datetime64[ns] as their standard time.

I am fixing this issue as a part of Pycon India conference

I am gonna work on this issue.

Navigation Menu

Repository navigation

DOC: Inconsistencies in converting array with `datetime` values into object array #19782

Reproducing code example:

NumPy/Python version information:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

DOC: Inconsistencies in converting array with datetime values into object array #19782

Description

Reproducing code example:

NumPy/Python version information:

Activity

Navaneethan2503 commented on Aug 30, 2021

BvB93 commented on Aug 30, 2021

hec10r commented on Aug 30, 2021

BvB93 commented on Aug 31, 2021

hec10r commented on Aug 31, 2021

charris commented on Aug 31, 2021

MrinaliniTh commented on Sep 20, 2021

pojaghi commented on Sep 16, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

DOC: Inconsistencies in converting array with `datetime` values into object array #19782