Repository navigation
datacenter.iers.org SSL: CERTIFICATE_VERIFY_FAILED (Column year failed to convert: invalid literal for int() with base 10: '<!') #12998
Description
UPDATE: Update to astropy v5.0.4 or later.
This sounds so familiar. Do you recall, @mhvk or @eteq ? This is causing remote data job failure.
urllib.error.URLError: <urlopen error Verification of TLS/SSL certificate at
https://datacenter.iers.org/data/9/finals2000A.all failed:
this can mean either the server is misconfigured or your local root CA certificates
are out-of-date; in the latter case this can usually be addressed by installing
the Python package "certifi" (see the documentation for astropy.utils.data.download_url)
or in both cases you can work around this by passing allow_insecure=True,
but only if you understand the implications; the original error was:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
Hostname mismatch, certificate is not valid for 'datacenter.iers.org'. (_ssl.c:1131)>
Activity
malynda commentedon Mar 25, 2022
Is the familiarity from when maia.usno.navy.mil disappeared?
pllim commentedon Mar 26, 2022
I contacted https://www.iers.org/IERS/EN/Organization/ProductCentres/EarthOrientationCentre/eoc.html 🤞
bjweiner commentedon Mar 27, 2022
This is causing astroplan to not work. There really needs to be a way to fall back gracefully to a lower time accuracy mode, rather than raising errors in astropy.iers that the user cannot resolve. I know that the code appears to contain fallbacks, but in practice they are not working. I can post the sequence of error messages if anyone finds it useful (not sure this thread is the right place).
cosmicpudding commentedon Mar 27, 2022
Have also encountered this as of today, and can't find a workaround either - like @bjweiner noted, there is meant to be an "auto_max_age" that can be set to None to eliminate the timeout of the table (tried both None and a high number):
from astropy.utils.iers import confconf.auto_max_age = 999conf.auto_download = Falsebut that parameter isn't doing anything (not sure if it is an astroplan or astropy issue). Is there anything that can be done in the meantime while the host server issues are in limbo? Thanks!
bjweiner commentedon Mar 27, 2022
I have an error message to report and a workaround. This is the error message I get:
I don't think conf.auto_max_age prevents this error. Next post has the fix.
bjweiner commentedon Mar 27, 2022
Of interest to @cosmicpudding if you were getting the IERSRangeError.
There are two places in astropy/time/core.py where the ut1_utc() method is called, as
Change both occurrences of this to
This will return the out-of-range code as a status rather than raising IERSRangeError and forcing the code to stop. I think this is a bug in astropy.time. (Edit to add: I wrote that at 2 AM, and have changed my opinion. I think this is a design issue with astropy.iers. In my opinion, astropy.iers should take care of all problems associated with failing to get an IERS file internally, and issue warnings to the code calling iers, not default to raising errors that cause the external code to stop.)
Should I open an issue for that or is this an appropriate place to discuss it? I think we need to develop a more robust testing strategy for dealing with old IERS files. A problem with these issues is that it's state dependent. If you have an in-date IERS file in the cache and simulate failing to download the IERS file, you won't get this IERSRangeError because you won't be out of range until some time later (a month?) when your file is out of date.
cosmicpudding commentedon Mar 27, 2022
Thanks @bjweiner, I was getting the same out of range error indeed! I can't change the system astropy where we're running the code, but managed to copy the system one to the local folder and made the changes you suggested there, specifically:
1653: #delta = iers_table.ut1_utc(jd1, jd2)1654: delta, status = iers_table.ut1_utc(jd1, jd2, return_status = True)and
1662: #delta = iers_table.ut1_utc(jd1_utc, jd2_utc)1663: delta, status = iers_table.ut1_utc(jd1_utc, jd2_utc, return_status = True)(It's possible we have an older version of astropy on the system, so I'm not sure the line numbers will completely match, but that's what they are anyway)
Thanks very much for the suggestion, great to have a workaround while (hopefully) IERS sorts their issues out!
pllim commentedon Mar 28, 2022
https://docs.astropy.org/en/stable/utils/iers.html#working-offline is supposed to work but there are caveats that I do not fully understand myself. If I remember correctly, they were last discussed in #12713 . Maybe @mhvk or @weaverba137 can clarify?
The logic is in
astropy/astropy/utils/iers/iers.py
Line 956 in 43f47a3
There is this note in the docstring FWIW
astropy/astropy/utils/iers/iers.py
Lines 885 to 888 in 43f47a3
weaverba137 commentedon Mar 28, 2022
See https://github.com/desihub/desiutil/blob/master/py/desiutil/iers.py for a function
freeze_iers()that handles this for the codes I work with.The trick is to call that function as early as possible in execution time, before any attempt to use or even import astropy.time.
TST: xfail test_iers_a_dl_mirror
pllim commentedon Mar 28, 2022
To those affected and no able to use offline mode, please see if #13004 works for you. Thanks!
bjweiner commentedon Mar 29, 2022
Hi, I really would like to have a conversation about how to design the iers module so this does not keep happening. I found that setting the iers.conf.auto_max_age = None parameter did not work to suppress this out of range error, contrary to the documentation.
I believe that the high precision IERS A data is needed strictly to track the UT1-UTC difference, which is always less than 1 second. 1 second of time does matter for accurately pointing a telescope, but not so much for making telescope schedules with e.g. astroplan. Every time the iers module fails irrecoverably, multiple people at multiple telescopes go into panic mode in the middle of the night. It is my opinion that iers should fall back to low precision mode by default rather than having to set extra variables to bypass critical errors. I don't know whether the best place to discuss this is here or on the astropy-dev slack workspace, but I would like to address this issue systematically rather than patching it around the edges.
pllim commentedon Mar 29, 2022
@bjweiner , please discuss at #9227 . Thanks!
[-]datacenter.iers.org SSL: CERTIFICATE_VERIFY_FAILED[/-][+]datacenter.iers.org SSL: CERTIFICATE_VERIFY_FAILED (Column year failed to convert: invalid literal for int() with base 10: '<!')[/+]pllim commentedon Apr 8, 2022
I think https://datacenter.iers.org/data/9/finals2000A.all just came back.