Ordering of leap second update can lead to import warnings

@taldcroft

After #9365 was merged, @taldcroft noticed that "In a clean build of master right now I'm seeing this:"

In [1]: from astropy.io import ascii
WARNING: leap-second auto-update failed due to the following exception: ImportError("cannot import name 'convert_numpy'",) [astropy.time.core]

???

Clearly, import order is important in ways it shouldn't be. I'll investigate but suggestions would be most welcome...

EDIT: reason it is tricky: many of the most common time formats need an up-to-date leap-second table (e.g., Time.now() uses TimeDatetime, which, like TimeISO* calls erfa.dtf2d, which needs leap second information. But the LeapSecond class subclasses QTable, so needs to import astropy.table. Table itself only imports Time locally, so is safe, but some parts of io.ascii import Time on top and therefore are not.

Possible solutions:

Make LeapSeconds a class independent of Table - which means also moving it out of iers. That means parsing the files outside of io.ascii and having very IERS-like configuration elsewhere; this seems a horrible duplication of work.
Ensure Time is imported only locally in io. But it seems rather annoying not to be able to count on being able to import such a fundamental class.
Ensure erfa is imported only locally in Time, iers, etc., and move the update back there. More than a little annoying, since used a lot in time.formats. Also, is a very basic package that should just be importable (and is not currently causing the problems).
Ensure the leap-second update is only done on first use of Time.

Option (4) may be the best. Should I do this with something in Time.__new__?

cc @taldcroft, @eteq

Good, I was just about to open this issue. In update_leap_seconds I change return 0 into raise and and there is a circular import that is otherwise just being ignored:

In [1]: import astropy.io.ascii
WARNING: leap-second auto-update failed due to the following exception: ImportError("cannot import name 'convert_numpy'",) [astropy.time.core]
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-0f9d21004cb9> in <module>()
----> 1 import astropy.io.ascii

~/git/astropy/astropy/io/ascii/__init__.py in <module>()
      5 
      6 
----> 7 from .core import (InconsistentTableError,
      8                    ParameterError,
      9                    NoType, StrType, NumType, FloatType, IntType, AllType,

~/git/astropy/astropy/io/ascii/core.py in <module>()
     28 from astropy.utils.exceptions import AstropyWarning
     29 
---> 30 from astropy.table import Table
     31 from astropy.utils.data import get_readable_fileobj
     32 from . import connect

~/git/astropy/astropy/table/__init__.py in <module>()
     57     from .jsviewer import JSViewer
     58     from astropy.io.ascii import connect
---> 59     from astropy.io.fits import connect
     60     from astropy.io.misc import connect
     61     from astropy.io.votable import connect

~/git/astropy/astropy/io/fits/connect.py in <module>()
     11 from astropy.table import Table, serialize, meta, Column, MaskedColumn
     12 from astropy.table.table import has_info_class
---> 13 from astropy.time import Time
     14 from astropy.utils.exceptions import AstropyUserWarning
     15 from astropy.utils.data_info import MixinInfo, serialize_context_as

~/git/astropy/astropy/time/__init__.py in <module>()
      4 
      5 
----> 6 update_leap_seconds()

~/git/astropy/astropy/time/core.py in update_leap_seconds(files)
   2411     """
   2412     try:
-> 2413         from astropy.utils import iers
   2414 
   2415         table = iers.LeapSeconds.auto_open(files)

~/git/astropy/astropy/utils/iers/__init__.py in <module>()
      1 
----> 2 from .iers import *

~/git/astropy/astropy/utils/iers/iers.py in <module>()
     24 from astropy import _erfa as erfa
     25 from astropy import config as _config
---> 26 from astropy.io.ascii import convert_numpy
     27 from astropy import units as u
     28 from astropy.table import QTable, MaskedColumn

ImportError: cannot import name 'convert_numpy'

I made that convert_numpy import local to avoid the circular import and something else popped up. It seems to be downloading a leap second file every time I try to import io.ascii.

(astropy) neptune$ git diff
diff --git a/astropy/utils/iers/iers.py b/astropy/utils/iers/iers.py
index b10fcbc21..5675f5828 100644
--- a/astropy/utils/iers/iers.py
+++ b/astropy/utils/iers/iers.py
@@ -23,7 +23,7 @@ import numpy as np
 from astropy.time import Time, TimeDelta
 from astropy import _erfa as erfa
 from astropy import config as _config
-from astropy.io.ascii import convert_numpy
+# from astropy.io.ascii import convert_numpy
 from astropy import units as u
 from astropy.table import QTable, MaskedColumn
 from astropy.utils.data import (get_pkg_data_filename, clear_download_cache,
@@ -1096,6 +1096,7 @@ class LeapSeconds(QTable):
         The file *must* contain the expiration date in a comment line, like
         '# File expires on:  28 June 2020'
         """
+        from astropy.io.ascii import convert_numpy
         names = ['ntp_seconds', 'tai_utc', 'comment', 'day', 'month', 'year']
         # Note: ntp_seconds does not fit in 32 bit, so causes problems on
         # 32-bit systems without the np.int64 converter.
(astropy) neptune$ 
(astropy) neptune$ 
(astropy) neptune$ ipython
Python 3.6.7 |Anaconda, Inc.| (default, Oct 23 2018, 14:01:38) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import astropy.io.ascii
/Users/aldcroft/git/astropy/astropy/table/column.py:988: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  result = getattr(super(), op)(other)
Downloading https://hpiers.obspm.fr/iers/bul/bulc/Leap_Second.dat
|=======================================================================================| 1.3k/1.3k (100.00%)         0s
Downloading https://www.ietf.org/timezones/data/leap-seconds.list [Done]
WARNING: IERSStaleWarning: leap-second file is expired. [astropy.utils.iers.iers]

Strange. And what is that FutureWarning doing there...

I think there must be another import problem being shielded by a try/except...

I have a solution along the lines of (4); see #9491

p.s. Am fairly sure the stale warning is due to hidden import problems, since the update itself is very well tested.

Navigation Menu

Repository navigation

Ordering of leap second update can lead to import warnings #9490

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Ordering of leap second update can lead to import warnings #9490

Description

Activity

taldcroft commented on Oct 27, 2019

taldcroft commented on Oct 27, 2019

mhvk commented on Oct 27, 2019

mhvk commented on Oct 27, 2019

mhvk commented on Oct 27, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions