Repository navigation
datetime -> Time accuracy problems #9579
Description
Description
Although python's datetime objects support microsecond resolution, and Time objects support sub-nanosecond resolution, conversion and arithmetic between the two can be off by tens of microseconds. (By design datetime objects do not support leap seconds, so UTC cannot be expected to work. The problems described here are for non-UTC timescales like TAI.)
Expected behavior
Computing the difference between two datetime objects then converting to a TimeDelta should give the same result as converting the datetime objects to Time objects and subtracting, to within a couple of microseconds.
Actual behavior
In fact PR #9532 demonstrates several examples where the difference exceeds 20 microseconds.
Steps to Reproduce
def test_datetime_difference_agrees_with_timedelta_no_hypothesis():
scale = "tai"
dt1 = datetime(1235, 1, 1, 0, 0)
dt2 = datetime(9950, 1, 1, 0, 0, 0, 890773)
t1 = Time(dt1, scale=scale)
t2 = Time(dt2, scale=scale)
assert(abs((t2-t1) - TimeDelta(dt2-dt1, scale=scale)) < 25*u.us)System Details
Python 3.7.3 (default, Apr 3 2019, 05:39:12)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform; print(platform.platform())
Linux-5.0.0-32-generic-x86_64-with-Ubuntu-19.04-disco
>>> import sys; print("Python", sys.version)
Python 3.7.3 (default, Apr 3 2019, 05:39:12)
[GCC 8.3.0]
>>> import numpy; print("Numpy", numpy.__version__)
Numpy 1.17.1
>>> import scipy; print("Scipy", scipy.__version__)
Scipy 1.3.1
>>> import astropy; print("astropy", astropy.__version__)
astropy 4.0.dev26780
Activity
mhvk commentedon Nov 10, 2019
No time right now to fix it, but I did find the problem: it is in the
TimeDeltacreation - theTimeDeltaDatetimeclass is based ontimedelta.total_seconds()which returns a double, hence the loss of precision (for this case with a very large number of days). Instead it should combine thetimedelta.{days, seconds, microseconds}attributes. Similarly, thevalueproperty should properly initialize usingdays,seconds, andmicroseconds. This should be a very easy fix, so labeling it accordingly.bmorris3 commentedon Nov 11, 2019
@mhvk – I'm working on this and I've I implemented the first part of your suggestion (getting rid of
total_seconds), but I'm not sure about what should happen tovalue– how does one take the JD and properly construct adatetime.timedeltausingdays,seconds,microseconds? Thanks!mhvk commentedon Nov 11, 2019
@bmorris3 - I haven't thought too hard, and it depends a little what
timedeltadoes internally, butjd1is the integer part of the JD difference, so is in days, and then one can constructsecondsandmicrosecondsfrom jd2 using something likeaarchiba commentedon Nov 11, 2019
Not to be difficult, but I'd like to point out that we were just talking about how we do not guarantee that jd1 and jd2 are related in this way. And in fact as a result of bug #9533 jd1 is not always an integer. For this code, it's probably safest to run
jd1, jd2 = day_frac(jd1, jd2)first.mhvk commentedon Nov 11, 2019
Actually,
timedeltadoes the addition of days and seconds internally anyway. At the precision needed here, I'd probably dobmorris3 commentedon Nov 11, 2019
Thanks for the tips, but neither tip seems to help make the proposed test pass. I think I'm missing something.
pllim commentedon Nov 11, 2019
@mhvk , "package-novice" might be a little misleading here, as the solution is only obvious when one already know the code base a little. As someone who doesn't touch the
timesub-package, I glanced at the conversation and I didn't get a clear picture on where to start, so I re-labeled it to "package-intermediate". FYI.Introduce non-hypothesis example for use in astropy#9579
Fix datetime accuracy
Add hypothesis testing for Time precision
aarchiba commentedon Nov 26, 2019
Indeed I think the subtlety is that your conversion, in both directions, needs to make full use of jd1 and jd2.
Add hypothesis testing for Time precision
Fix datetime accuracy