Possible improvements to how IERS is handled

@aarchiba

In #9204, @aarchiba wrote a comment that deserves to be its own issue:

Thinking about user desires:

User wants a specific IERS A table, or IERS B table, or merged version, for reproducibility.
User wants some kind of IERS data and has no connectivity.
User wants an IERS A or B table that covers all the data in some particular data set.
User wants the best available data that covers their data set.
User wants IERS data to "just work" for reasonable times, past present and near future.
User doesn't actually care much about accuracy but needs a stable set of IERS data for simulations possibly extending quite far into the future. (Observation planning, or generating par files for future observations of the triple system, for example.)
User wants all astropy things that use IERS tables to use the same set of tables, whether autoupdated or not: having old the IERS data change partway through a computation because you requested an observatory position past the end of the cached table is liable to lead to some hair-tearing bugs that affect scientific results.

I'd like to see:

A way to merge specific IERS A and IERS B tables to get a best-available combination.
A way to get up-to-date-enough IERS A and IERS B tables (PINT needs IERS B specifically to agree with TEMPO2), preferably without user effort beyond perhaps making one request for all times up front.
A way to have a "best-effort" IERS table - if the network is up, a merged version of auto-updated IERS A and B. Graceful degradation through the most recent A and B tables that exist in the cache to the plain B table distributed with astropy if the network is not available.
A way to tell astropy to use specific table versions for either everything, or just everything inside this with block.

These four seem to cover all the use cases that sprang to my mind. Only the last, the context management, seems tricky to me, and that only if multithreading is to work right.

I think we can get quite a bit on the way by:

Having the IERS file used throughout astropy be a ScienceState object (see Let IERS table used in time/coordinates determined by science state #9244);
Extending auto-update to IERS-B (i.e., maybe have IERS_A_Auto and IERS_B_Auto classes); (see download_file with cache=True uses cache unconditionally #3961 for general caching issues...)
Use List of mirror URLs for IERS download astropy-iers-data#20
Allow explicit merging of two IERS-B and IERS-A files.
(edit) Does this still belong in utils? astropy.iers might be more logical
What about Post-leap-second update to do list #9479 ?
What about IERS warning during astropy.coordinates for dates that should be covered by IERS data #10219 ?

@aarchiba

@aarchiba - see #9244 for the introduction of a new iers.earth_reference_system science state class that allows one to set the IERS table used for all calculations. Arguably, this is the most important improvement, since PINT can now prepare a table anyway it wants (though the other items are still worth fixing/considering)

@eteq

The particular issue of IERS and reproducibility has come up here at Petabytes 3 this week. @eteq has suggested the following straw-person API to at least provide a reproducible mode:

from astropy.utils.iers import IERS_from_the_past

with IERS_from_the_past('2018-1-1'):
	coord.transform_to(AltAz(...))

As far as I can think through it, this would work. I do worry a bit about the scalability: if one's code has 7 different sections that use magically-downloaded files from 5 different services, it would be pretty tedious to edit everything to use the right reproducibility-enabled versions of the API calls. Composability is also an issue: what if my code calls a third-party module that uses the IERS table, but that module doesn't have the from_the_past mode?

@pkgw

@pkgw - I think the new ScienceState does solve this (at least within astropy): you'd do

iers.earth_reference_system.set(IERS_from_the_past(...))

(possibly as a context manager) and then all of astropy would use that table by default. Of course, whether or not other libraries will follow and start getting their default IERS table from earth_reference_system, I cannot guarantee...

In any case, step 1 might be to open some PR to allow the IERS classes to find older files based on date.

Following up on #10387 (comment), that hard-to-track-down test failure brings some new urgency to this issue, and it is probably good to describe why:

Our current IERS_Auto tries to give a "best of both worlds" by combining the most up to date IERS_A values and predictions for present and future with presumably more secure IERS_B values for the past. IERS_A contains "_B" values itself, which are presumably from IERS_B but in actuality are often slightly different from IERS_B (which, certainly contrary to what we thought initially, changes with time). So, we have code that updates the "_B" values with actual ones from IERS_B.

What it currently does not do, however, is add values in case the IERS_B has newer ones than present in the IERS_A file (which led to a hard-to-trace test failure in #10387). Should this be changed?

More generally, should we be messing with the IERS_A file at all? Eg., it is possible that the "*_B" values are actually from a newer file than the IERS_B file astropy has bundled. A suggestion might be:

Define IERS_A_Auto and IERS_B_Auto, which just ensure that the respective IERS files are up to date.
Define IERS_Auto` as the mix of the two now guaranteed-up-to-date files.

This would seem consistent with the wish list on top.

Hi, I understand that there are some complex issues here and that one does not want to design the IERS module so as to rule out future development of improved IERS tables, tracking for reproducibility, and so on. However, I want to raise a more basic design issue (I think). astropy.iers is frequently called by other astropy (or astroplan) modules. Many, I'd guess most, astropy "users" do not call iers themselves. So when iers throws an error, the user cannot easily fix it by trapping it in user code. (See #12998 for an example where I had to patch astropy.time to bypass the latest IERS outage and astropy.iers error.) Because astropy.iers failures result in a fundamental inability to tell the time, this affects critical systems even if they don't need high precision.

I want to advocate that astropy.iers should default to a "Just Work" behavior that is robust and can always return at least a low precision value and a warning, regardless of network outages, rather than throwing an error. While users who need to insure the highest accuracy or extra measures to insure reproducibility would set this behavior by some optional variable.

Can we also add a possibility for user-configurable mirror? At the moment two mirrors are hardcoded and providing a different one requires source code modification.
Also, CDDIS (which was a default option, but didn't work for some time due to SSL/TLS encryption) provides a python snippet working with their FTP archive here:
https://cddis.nasa.gov/Data_and_Derived_Products/CDDIS_Archive_Access.html
In case we are planning to use CDDIS, an overloaded method to download from it is also required

@weaverba137

Actually, they are configurable:

astropy/astropy/utils/iers/iers.py

Lines 126 to 131 in 910f2f7

    
           iers_auto_url = _config.ConfigItem( 
        
               IERS_A_URL, 
        
               'URL for auto-downloading IERS file data.') 
        
           iers_auto_url_mirror = _config.ConfigItem( 
        
               IERS_A_URL_MIRROR, 
        
               'Mirror URL for auto-downloading IERS file data.')

But as @weaverba137 mentioned above, the trick is to get all that to kick in before some internal import tries to access the leap seconds table.

Maybe try setting it in your ~/.astropy/config/astropy.cfg? See https://docs.astropy.org/en/stable/config/index.html

I'm trying to follow the threads here and not quite succeeding since there is a lot going on in multiple issues.

As a start, can someone post a reproducible example that shows a problem (there may be more than one). The example to reproduce in #13024 does not fail for me on current main. Perhaps an example with setting all the servers to addresses that will certainly fail would provide a reproducible example?

Part of the problem seems to be with servers that come and go. The desire to decouple from that uncertainty is well-founded, and is supposed to be handled with the conf.auto_download option. If that isn't working as expected or is not an adequate solution then that seems like a concrete starting point for debugging / discussion.

There also should be no attempt to download the leap seconds table until an actual conversion to/from UTC is done. The astropy core package should not be doing any such conversion on import of any subpackage. If that is not the case then that is a bug we can fix.

@taldcroft

Hi @taldcroft, A problem here is that the IERS file bugs are state dependent. In addition to the obvious dependency of whether the server is up or down, the IERS A file is downloaded and cached for some period of time. So if you have a recent IERS A file in the cache and you run some code that uses IERS, it won't try to download the file and you won't get an error even if the IERS servers are all down.

As a possible example, try this:

from astropy.time import Time
Time('2022-05-01', scale='utc').ut1

Right now, I get the error:

Downloading ftp://anonymous:mail%[email protected]/pub/products/iers/finals2000A.all from https://datacenter.iers.org/data/9/finals2000A.all
|======================================================================| 2.1k/2.1k (100.00%) 0s
Traceback (most recent call last):
File "/home/mmtop/anaconda3/lib/python3.8/site-packages/astropy/io/ascii/core.py", line 969, in _convert_vals
converter_func, converter_type = col.converters[0]
IndexError: list index out of range

and then another chain of errors culminating in

ValueError: Column year failed to convert: invalid literal for int() with base 10: '<!'

This is because https://datacenter.iers.org/data/9/finals2000A.all is returning a "page not available" rather than a table. If you don't get this error, try running:

from astropy.utils.data import clear_download_cache
clear_download_cache()
Time('2022-05-01', scale='utc').ut1

I'm not 100% sure this will reproduce the error, but I think it should. It may also cripple your system by getting rid of a working IERS A file.

Finally, I think some re-engineering is necessary so that even if a server is down or the IERS A file is invalid, a simple time conversion like the above defaults to returning a low precision value (within 1 second) rather than an error.

Edited: The version of astropy used here was 4.0, although I think later versions will behave the same.

If we are reading HTTP error messages as table content does that imply that astropy.utils.data.download_file is missing an HTTP status check somewhere?

@timj

an idea that @timj brought up on slack is to have astropy do it's own mirroring of the IERS data that it uses. that way the handling of the download mirrors and data validation can happen outside of astropy itself and be more responsive to changes. astropy itself would then point to the tested/validated files hosted on github.

If we are reading HTTP error messages as table content does that imply that astropy.utils.data.download_file is missing an HTTP status check somewhere?

It's possible we missed a check, though we use urllib for the actual downloads, but it's also possible that a web server begins returning an error page but not actually sending an HTTP error code.

How many of the items are no longer relevant now that we have a separate astropy-iers-data package?

I think this one also has mostly become one of documentation in astropy.utils.iers. Though the lack of complaints suggests that perhaps one can just close this one too. Let me just be bold and do that...

Navigation Menu

Repository navigation

Possible improvements to how IERS is handled #9227

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	iers_auto_url = _config.ConfigItem(
	IERS_A_URL,
	'URL for auto-downloading IERS file data.')
	iers_auto_url_mirror = _config.ConfigItem(
	IERS_A_URL_MIRROR,
	'Mirror URL for auto-downloading IERS file data.')

Possible improvements to how IERS is handled #9227

Description

Activity

mhvk commented on Sep 16, 2019

pkgw commented on Nov 8, 2019

mhvk commented on Nov 10, 2019

mhvk commented on May 24, 2020

bjweiner commented on Mar 31, 2022

mexanick commented on Mar 31, 2022

pllim commented on Mar 31, 2022

taldcroft commented on Mar 31, 2022

bjweiner commented on Apr 2, 2022

timj commented on May 6, 2022

tepickering commented on May 6, 2022

aarchiba commented on May 25, 2022

pllim commented on Oct 27, 2023

mhvk commented on Oct 28, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions