[Python] Pyarrow allows for unsafe conversions of datetime objects to timestamp nanoseconds

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Minor
    • 2.0.0
    • Affects Version/s: 0.17.1, 1.0.0
    • Component/s: Python
    • Environment:
      OS: MacOSX Catalina
      Python Version: 3.7

      Hi, 

      In parquet, I want to store date values as timestamp format with nanoseconds precision. This works fine with most dates except those past pandas.Timestamp.max: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Timestamp.max.html.

      I was expecting some exception to be raised (like in Pandas), however this did not happen and the value was processed incorrectly. Note that this is with safe=True. Can this please be looked into? Thanks

      Example Code:

      pa.array([datetime(2262,4,12)], type=pa.timestamp("ns"))
      {{}}

      Return:

      [
      {{ 1677-09-21 00:25:26.290448384}}
      ]

            Assignee:
            Joris Van den Bossche
            Reporter:
            Joshua Lay (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2.5h
                2.5h