Skip to content

IERS data not downloading #13024

Closed
Closed

Description

@DrPhilEvans

Description

astropy crashes when downloading IERS data. The file it retrieves is actually an HTML page giving an error saying "This service is not available", but astropy tries to parse it as the expected file, and failed.

Expected behavior

Successful return of astropy time functions. If the downloaded IERS file is wrong / not conforming to spec, a different source should be used, instead of throwing an error.

Actual behavior

Here is the output from my simple test:

In [1]: from astropy.time import Time
   ...: Time.now().ut1
Downloading https://hpiers.obspm.fr/iers/bul/bulc/Leap_Second.dat
|======================================================================================================================================================================| 1.3k/1.3k (100.00%)         0s
Downloading ftp://anonymous:mail%[email protected]/pub/products/iers/finals2000A.all from https://datacenter.iers.org/data/9/finals2000A.all
|======================================================================================================================================================================| 2.1k/2.1k (100.00%)         0s
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-50505784fe6d> in <module>
      1 from astropy.time import Time
----> 2 Time.now().ut1

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/time/core.py in __getattr__(self, attr)
   1334                 else:
   1335                     tm = self.replicate()
-> 1336                     tm._set_scale(attr)
   1337                     if tm.shape:
   1338                         # Prevent future modification of cached array-like object

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/time/core.py in _set_scale(self, scale)
    582                     pass
    583                 else:
--> 584                     args.append(get_dt(jd1, jd2))
    585                     break
    586 

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/time/core.py in _get_delta_ut1_utc(self, jd1, jd2)
   2039         if not hasattr(self, '_delta_ut1_utc'):
   2040             from astropy.utils.iers import earth_orientation_table
-> 2041             iers_table = earth_orientation_table.get()
   2042             # jd1, jd2 are normally set (see above), except if delta_ut1_utc
   2043             # is access directly; ensure we behave as expected for that case

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/utils/state.py in get(cls)
     38         Get the current science state value.
     39         """
---> 40         return cls.validate(cls._value)
     41 
     42     @classmethod

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/utils/iers/iers.py in validate(cls, value)
    860     def validate(cls, value):
    861         if value is None:
--> 862             value = IERS_Auto.open()
    863         if not isinstance(value, IERS):
    864             raise ValueError("earth_orientation_table requires an IERS Table.")

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/utils/iers/iers.py in open(cls)
    692             return cls.iers_table
    693 
--> 694         cls.iers_table = cls.read(file=filename)
    695         cls.iers_table.meta['data_url'] = all_urls[0]
    696 

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/utils/iers/iers.py in read(cls, file, readme)
    545             readme = IERS_A_README
    546 
--> 547         iers_a = super().read(file, format='cds', readme=readme)
    548 
    549         # Combine the A and B data for UT1-UTC and PM columns

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/table/connect.py in __call__(self, *args, **kwargs)
     60         descriptions = kwargs.pop('descriptions', None)
     61 
---> 62         out = self.registry.read(cls, *args, **kwargs)
     63 
     64         # For some readers (e.g., ascii.ecsv), the returned `out` class is not

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/io/registry/core.py in read(self, cls, format, cache, *args, **kwargs)
    197 
    198             reader = self.get_reader(format, cls)
--> 199             data = reader(*args, **kwargs)
    200 
    201             if not isinstance(data, cls):

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/io/ascii/connect.py in io_read(format, filename, **kwargs)
     16         format = re.sub(r'^ascii\.', '', format)
     17         kwargs['format'] = format
---> 18     return read(filename, **kwargs)
     19 
     20 

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/io/ascii/ui.py in read(table, guess, **kwargs)
    374         else:
    375             reader = get_reader(**new_kwargs)
--> 376             dat = reader.read(table)
    377             _read_trace.append({'kwargs': copy.deepcopy(new_kwargs),
    378                                 'Reader': reader.__class__,

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/io/ascii/cds.py in read(self, table)
    341                     return table
    342         else:
--> 343             return super().read(table)

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/io/ascii/core.py in read(self, table)
   1379                                      self.include_names, self.exclude_names)
   1380 
-> 1381         table = self.outputter(self.header.cols, self.meta)
   1382         self.cols = self.header.cols
   1383 

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/io/ascii/core.py in __call__(self, cols, meta)
   1115         # Sets col.data to numpy array and col.type to io.ascii Type class (e.g.
   1116         # FloatType) for each col.
-> 1117         self._convert_vals(cols)
   1118 
   1119         t_cols = [numpy.ma.MaskedArray(x.data, mask=x.mask)

~/soft/Python-3.8.1/lib/python3.8/site-packages/astropy/io/ascii/core.py in _convert_vals(self, cols)
   1058                 # If there are no converters left here then fail.
   1059                 if not col.converters:
-> 1060                     raise ValueError(f'Column {col.name} failed to convert: {last_err}')
   1061 
   1062                 converter_func, converter_type = col.converters[0]

ValueError: Column year failed to convert: invalid literal for int() with base 10: '<!'

Steps to Reproduce

  • Remove your .astropy directory (i.e. so nothing is cached)
  • Open ipython:
from astropy.time import Time
Time.now().ut1

System Details

Linux-4.15.0-171-generic-x86_64-with-glibc2.27
Python 3.8.1 (default, Jun 16 2021, 12:02:47) 
[GCC 7.5.0]
Numpy 1.20.3
pyerfa 2.0.0
astropy 5.0
Matplotlib 3.4.2

(scipy not installed)

Problem also occurs on a CentOS7 server:

Linux-3.10.0-1160.42.2.el7.x86_64-x86_64-with-glibc2.2.5
Python 3.8.1 (default, Feb 20 2020, 08:38:18) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
Numpy 1.22.2
pyerfa 2.0.0.1
astropy 5.0.1
Scipy 1.8.0
Matplotlib 3.5.1

Activity

  1. github-actions commented on Mar 31, 2022

    @github-actions
    Contributor

    Welcome to Astropy 👋 and thank you for your first issue!

    A project member will respond to you as soon as possible; in the meantime, please double-check the guidelines for submitting issues and make sure you've provided the requested details.

    GitHub issues in the Astropy repository are used to track bug reports and feature requests; If your issue poses a question about how to use Astropy, please instead raise your question in the Astropy Discourse user forum and close this issue.

    If you feel that this issue has not been responded to in a timely manner, please leave a comment mentioning our software support engineer @embray, or send a message directly to the development mailing list. If the issue is urgent or sensitive in nature (e.g., a security vulnerability) please send an e-mail directly to the private e-mail [email protected].

  2. pllim commented on Mar 31, 2022

    @pllim
    Member

    This is a known issue and has been fixed in dev but not released... Please see #12998 . Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions