IERSRangeError after downloading table from USNO

Description

Recent dates don't seem to be included in the IERS table when I force a fresh download

Expected behavior

I expected to convert a recent date

Actual behavior

astropy.utils.iers.iers.IERSRangeError: (some) times are outside of range covered by IERS table.

Steps to Reproduce

# Put your Python code snippet here.
from astropy.utils.data import download_file
from astropy.time import Time, TimeDelta
from astropy.utils import iers
url = 'https://maia.usno.navy.mil/ser7/finals2000A.all'
filename = download_file(url, cache=False)  # Force download of latest copy
dat = iers.IERS_Auto.read(file=filename)
iers.IERS.iers_table = dat
iers.conf.auto_download = False
(Time.now()-TimeDelta(3)).ut1 #Raises Error
# astropy.utils.iers.iers.IERSRangeError: (some) times are outside of range covered by IERS table.
for i in range(1000):
    try:
        LastDate = (Time('2021-01-11')+TimeDelta(i)).ut1
    except iers.iers.IERSRangeError:
        print(Time('2021-01-11')+TimeDelta(i))
        break
# 2022-02-06 00:00:00.000 is the first date that cannot be converted

System Details

Python 3.9.12 (main, Apr 5 2022, 06:56:58)
[GCC 7.5.0]
Numpy 1.21.5
pyerfa 2.0.0
astropy 5.0.4
Scipy 1.7.3
Matplotlib 3.5.1

Since we do not control this data file, I have inquired with USNO and will post here if I hear back. Thanks for your patience!

Since we do not control this data file, I have inquired with USNO and will post here if I hear back. Thanks for your patience!

Thank you for contacting USNO for me. What really confuses me is that if I get the year, month and day from the final row of the table, it is a date about 1 year in the future:

from datetime import datetime, timedelta
from astropy.utils.data import download_file
from astropy.time import Time, TimeDelta
from astropy.utils import iers
url = 'https://maia.usno.navy.mil/ser7/finals2000A.all'
filename = download_file(url, cache=False)  # Force download of latest copy
dat = iers.IERS_Auto.read(file=filename)
Latest=datetime(2000+dat[-1]['year'],dat[-1]['month'],dat[-1]['day']).strftime('%Y-%m-%d')
print(Latest) #'2023-05-13'

The error only pops up when iers.conf.auto_download = False 🤔

@mhvk

This is smelling like a bug. The data is obviously there but somehow "invisible" when auto_download is set to False. My hunch is that the bug is somewhere here:

astropy/astropy/utils/iers/iers.py

Line 756 in 873d2c9

def _refresh_table_as_needed(self, mjd):

But I don't grok IERS or UT1 enough to see what is supposed to happen. Maybe @mhvk or @taldcroft have ideas?

Maybe your code is somehow grabbing IERS B. What if you do this? Does it go all the way to May 2023 before crashing?

iers_a = iers.IERS_A.open(iers.IERS_A_URL)
iers.earth_orientation_table.set(iers_a)

@pllim

Thanks! I think @pllim has the right work-around, but you've clearly found a bug: unless one sets earth_orientation_table explicitly (like @pllim suggested) any Time use would call IERS_Auto.open() and that starts with

astropy/astropy/utils/iers/iers.py

Lines 702 to 714 in 873d2c9

    
           if not conf.auto_download: 
        
               cls.iers_table = IERS_B.open() 
        
               return cls.iers_table 
        
           all_urls = (conf.iers_auto_url, conf.iers_auto_url_mirror) 
        
           if cls.iers_table is not None: 
        
               # If the URL has changed, we need to redownload the file, so we 
        
               # should ignore the internally cached version. 
        
               if cls.iers_table.meta.get('data_url') in all_urls: 
        
                   return cls.iers_table

You see that with auto_download=False it forces the IERS_B table, even though a good table is already set. Logically, it should just use the table present if one is there.

Would you be willing to make a quick PR? A test could use your example, except perhaps with IERS_A_EXCERPT instead of downloading the full file (or it could be skipped if not HAS_IERS_A; see iers/tests/test_iers.py)

Setting earth_orientation_table explicitly works for me. Thank you!

@Warren-Porter

@Warren-Porter , glad the workaround worked for you. Are you still interested to submit a PR to fix the underlying bug as mentioned in #13227 (comment) ? 😸

@mhvk

We encountered this same bug in the different context of puzzling CI errors. astropy's conftest.py naturally sets iers.conf.auto_download = False (for offline testing), but as noted by @mhvk, this has the unintuitive behavior of disregarding the IERS A table. For example, in the latest astropy-iers-data release at the time of this writing, IERS A has historical data up to February 13, but IERS B has data only up to January 16. Thus, a seemingly innocuous time conversion of, say, February 1 raises a warning on CI but does not raise a warning for a user with the default setting of iers.conf.auto_download = True.

Here's a minimal test example, illustrating the silliness that a just-downloaded IERS A table is subsequently ignored if iers.conf.auto_download = False:

>>> from astropy.time import Time
>>> Time('2024-02-01').ut1
Downloading https://datacenter.iers.org/data/9/finals2000A.all
|===========================================| 3.5M/3.5M (100.00%)         6s
<Time object: scale='ut1' format='iso' value=2024-02-01 00:00:00.004>

>>> from astropy.utils import iers
>>> iers.conf.auto_download = False

>>> Time('2024-02-01').ut1
Traceback (most recent call last):
...
astropy.utils.iers.iers.IERSRangeError: (some) times are outside of range covered by IERS table. Cannot convert with full accuracy. To allow conversion with degraded accuracy set astropy.utils.iers.conf.iers_degraded_accuracy to "warn" or "silent". For more information about setting this configuration parameter or controlling its value globally, see the Astropy configuration system documentation https://docs.astropy.org/en/stable/config/index.html.

Curiously, the documentation for the configuration parameter correctly states this behavior of looking at only IERS B, but the guide page for astropy.utils.iers is incorrect and states that IERS A will also be used in addition to IERS B even if auto_download = False.

Indeed, this clearly is a bug - fix like in my comment most welcome -- though it is now clear one would also need to adjust the documentation of the configuration parameter!

Okay, I did a bit of archaeology, and this logic unsurprisingly traces back eight years ago to when auto-downloading of IERS A was implemented (#4436), but there was no expectation that IERS A would get bundled. So, auto_download = False served two purposes in order to make Astropy 1.2 behave like 1.1:

Do not trigger downloads
Use the bundled table (IERS B) even if IERS A has been downloaded

Fast forward to the present day, with IERS A now bundled (#14819), we should definitely fix the bug that auto_download = False ignores IERS A. The question on my mind is whether auto_download = False should still ignore a downloaded IERS A. I see value in forcing code to use the bundled tables, but I think the keyword name is unfortunate (versus something clearer like use_bundled = True).

Is there any real need of ignoring IERS A when it is available anyway? 👀

The argument for ignoring a downloaded IERS A is reproducibility. If auto_download = False forces code to use the bundled IERS A instead of any downloaded IERS A, then it's possible to reproduce a computation by knowing only the version of astropy-iers-data.

I think we should ensure that auto_download just does what its name suggest - not download anything. If people want a specific IERS file, they can just open that file explicitly. (But clearly, this will be not just a bug fix but a API change, even if implicit).

Navigation Menu

Repository navigation

IERSRangeError after downloading table from USNO #13227

Description

Expected behavior

Actual behavior

Steps to Reproduce

System Details

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if not conf.auto_download:
	cls.iers_table = IERS_B.open()
	return cls.iers_table

	all_urls = (conf.iers_auto_url, conf.iers_auto_url_mirror)

	if cls.iers_table is not None:

	# If the URL has changed, we need to redownload the file, so we
	# should ignore the internally cached version.

	if cls.iers_table.meta.get('data_url') in all_urls:
	return cls.iers_table

IERSRangeError after downloading table from USNO #13227

Description

Description

Expected behavior

Actual behavior

Steps to Reproduce

System Details

Activity

pllim commented on May 6, 2022

Warren-Porter commented on May 6, 2022

pllim commented on May 6, 2022

pllim commented on May 6, 2022

pllim commented on May 6, 2022

mhvk commented on May 6, 2022

Warren-Porter commented on May 7, 2022

pllim commented on May 7, 2022

ayshih commented on Mar 9, 2024

mhvk commented on Mar 9, 2024

ayshih commented on Mar 11, 2024

pllim commented on Mar 11, 2024

ayshih commented on Mar 11, 2024

mhvk commented on Mar 11, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions