Skip to content

TST: 201 test failures in coordinates in master branch #11307

Closed

Description

@pllim

First seen after merging a seemingly unrelated #11301 and it also coincided with Numpy 1.20.1 release (which is a bug fix release that should not change any correct behavior? See https://github.com/numpy/numpy/releases/tag/v1.20.1). This is blocking unrelated PRs big time.

Example log: https://github.com/astropy/astropy/actions/runs/548211298

...
2021-02-08T14:19:08.2049535Z ________________________ test_icrs_consistency[icoo199] ________________________
2021-02-08T14:19:08.2049909Z 
2021-02-08T14:19:08.2050407Z icoo = <SkyCoord (ICRS): (ra, dec, distance) in (deg, deg, km)
2021-02-08T14:19:08.2050912Z     (228.64441847, 7.8492488, 29817.50654624)>
2021-02-08T14:19:08.2051163Z 
2021-02-08T14:19:08.2051862Z     @pytest.mark.parametrize('icoo', icrs_coords)
2021-02-08T14:19:08.2052536Z     def test_icrs_consistency(icoo):
2021-02-08T14:19:08.2052956Z         """
2021-02-08T14:19:08.2053656Z         Check ICRS<->AltAz for consistency with ICRS<->CIRS<->AltAz
2021-02-08T14:19:08.2054159Z     
2021-02-08T14:19:08.2054888Z         The latter is extensively tested in test_intermediate_transformations.py
2021-02-08T14:19:08.2055564Z         """
2021-02-08T14:19:08.2056085Z         observer = EarthLocation(28*u.deg, 23*u.deg, height=2000.*u.km)
2021-02-08T14:19:08.2056829Z         obstime = Time('J2010')
2021-02-08T14:19:08.2057394Z         aa_frame = AltAz(obstime=obstime, location=observer)
2021-02-08T14:19:08.2057900Z     
2021-02-08T14:19:08.2058283Z         # check we are going direct!
2021-02-08T14:19:08.2058939Z         trans = frame_transform_graph.get_transform(ICRS, AltAz).transforms
2021-02-08T14:19:08.2059572Z         assert(len(trans) == 1)
2021-02-08T14:19:08.2059931Z     
2021-02-08T14:19:08.2060644Z         # check that ICRS-AltAz and ICRS->CIRS->AltAz are consistent
2021-02-08T14:19:08.2061289Z         aa1 = icoo.transform_to(aa_frame)
2021-02-08T14:19:08.2061921Z         aa2 = icoo.transform_to(CIRS()).transform_to(aa_frame)
2021-02-08T14:19:08.2062610Z >       assert_allclose(aa1.separation_3d(aa2), 0*u.mm, atol=1*u.mm)
2021-02-08T14:19:08.2063030Z 
2021-02-08T14:19:08.2064053Z ../../.tox/py39-test/lib/python3.9/site-packages/astropy/coordinates/tests/test_icrs_observed_transformations.py:39: 
2021-02-08T14:19:08.2064913Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2021-02-08T14:19:08.2065187Z 
2021-02-08T14:19:08.2065871Z actual = <Distance 3.16026045e-05 km>, desired = <Quantity 0. mm>, rtol = 1e-07
2021-02-08T14:19:08.2066451Z atol = <Quantity 1. mm>, kwargs = {}
2021-02-08T14:19:08.2067502Z np = <module 'numpy' from '/home/runner/work/astropy/astropy/.tox/py39-test/lib/python3.9/site-packages/numpy/__init__.py'>
2021-02-08T14:19:08.2069884Z _unquantify_allclose_arguments = <function _unquantify_allclose_arguments at 0x7ff9cbd2f280>
2021-02-08T14:19:08.2070480Z 
2021-02-08T14:19:08.2071253Z     def assert_quantity_allclose(actual, desired, rtol=1.e-7, atol=None,
2021-02-08T14:19:08.2071871Z                                  **kwargs):
2021-02-08T14:19:08.2072219Z         """
2021-02-08T14:19:08.2072806Z         Raise an assertion if two objects are not equal up to desired tolerance.
2021-02-08T14:19:08.2073367Z     
2021-02-08T14:19:08.2074133Z         This is a :class:`~astropy.units.Quantity`-aware version of
2021-02-08T14:19:08.2074911Z         :func:`numpy.testing.assert_allclose`.
2021-02-08T14:19:08.2075426Z         """
2021-02-08T14:19:08.2075791Z         import numpy as np
2021-02-08T14:19:08.2076506Z         from astropy.units.quantity import _unquantify_allclose_arguments
2021-02-08T14:19:08.2077599Z >       np.testing.assert_allclose(*_unquantify_allclose_arguments(
2021-02-08T14:19:08.2078319Z             actual, desired, rtol, atol), **kwargs)
2021-02-08T14:19:08.2078855Z E       AssertionError: 
2021-02-08T14:19:08.2079579Z E       Not equal to tolerance rtol=1e-07, atol=1e-06
2021-02-08T14:19:08.2080041Z E       
2021-02-08T14:19:08.2080453Z E       Mismatched elements: 1 / 1 (100%)
2021-02-08T14:19:08.2081157Z E       Max absolute difference: 3.16026045e-05
2021-02-08T14:19:08.2081697Z E       Max relative difference: inf
2021-02-08T14:19:08.2082434Z E        x: array(3.16026e-05)
2021-02-08T14:19:08.2082821Z E        y: array(0.)
2021-02-08T14:19:08.2083074Z 
2021-02-08T14:19:08.2083948Z ../../.tox/py39-test/lib/python3.9/site-packages/astropy/tests/helper.py:491: AssertionError
2021-02-08T14:19:08.2084759Z _____________________________ test_against_pyephem _____________________________
2021-02-08T14:19:08.2085087Z 
2021-02-08T14:19:08.2085491Z     def test_against_pyephem():
2021-02-08T14:19:08.2086174Z         """Check that Astropy gives consistent results with one PyEphem example.
2021-02-08T14:19:08.2086757Z     
2021-02-08T14:19:08.2087333Z         PyEphem: https://rhodesmill.org/pyephem/
2021-02-08T14:19:08.2087902Z     
2021-02-08T14:19:08.2088331Z         See example input and output here:
2021-02-08T14:19:08.2088972Z         https://gist.github.com/zonca/1672906
2021-02-08T14:19:08.2090007Z         https://github.com/phn/pytpm/issues/2#issuecomment-3698679
2021-02-08T14:19:08.2090636Z         """
2021-02-08T14:19:08.2091267Z         obstime = Time('2011-09-18 08:50:00')
2021-02-08T14:19:08.2092017Z         location = EarthLocation(lon=Angle('-109d24m53.1s'),
2021-02-08T14:19:08.2092767Z                                  lat=Angle('33d41m46.0s'),
2021-02-08T14:19:08.2093223Z                                  height=30000. * u.m)
2021-02-08T14:19:08.2093843Z         # We are using the default pressure and temperature in PyEphem
2021-02-08T14:19:08.2094485Z         # relative_humidity = ?
2021-02-08T14:19:08.2094890Z         # obswl = ?
2021-02-08T14:19:08.2095446Z         altaz_frame = AltAz(obstime=obstime, location=location,
2021-02-08T14:19:08.2096147Z                             temperature=15 * u.deg_C, pressure=1.010 * u.bar)
2021-02-08T14:19:08.2096639Z     
2021-02-08T14:19:08.2097260Z         altaz = SkyCoord('6.8927d -60.7665d', frame=altaz_frame)
2021-02-08T14:19:08.2098036Z         radec_actual = altaz.transform_to('icrs')
2021-02-08T14:19:08.2098492Z     
2021-02-08T14:19:08.2099181Z         radec_expected = SkyCoord('196.497518d -4.569323d', frame='icrs')  # EPHEM
2021-02-08T14:19:08.2100102Z         # radec_expected = SkyCoord('196.496220d -4.569390d', frame='icrs')  # HORIZON
2021-02-08T14:19:08.2101059Z         distance = radec_actual.separation(radec_expected).to('arcsec')
2021-02-08T14:19:08.2101797Z         # TODO: why is this difference so large?
2021-02-08T14:19:08.2102328Z         # It currently is: 31.45187984720655 arcsec
2021-02-08T14:19:08.2102849Z         assert distance < 1e3 * u.arcsec
2021-02-08T14:19:08.2103253Z     
2021-02-08T14:19:08.2103830Z         # Add assert on current Astropy result so that we notice if something changes
2021-02-08T14:19:08.2104765Z         radec_expected = SkyCoord('196.495372d -4.560694d', frame='icrs')
2021-02-08T14:19:08.2105699Z         distance = radec_actual.separation(radec_expected).to('arcsec')
2021-02-08T14:19:08.2106376Z         # Current value: 0.0031402822944751997 arcsec
2021-02-08T14:19:08.2106868Z >       assert distance < 1 * u.arcsec
2021-02-08T14:19:08.2107420Z E       assert <Angle 521.28428171 arcsec> < (1 * Unit("arcsec"))
2021-02-08T14:19:08.2107953Z E        +  where Unit("arcsec") = u.arcsec
2021-02-08T14:19:08.2108275Z 
2021-02-08T14:19:08.2109279Z ../../.tox/py39-test/lib/python3.9/site-packages/astropy/coordinates/tests/accuracy/test_altaz_icrs.py:127: AssertionError
...
FAILED ../../.tox/py39-test/lib/python3.9/site-packages/astropy/coordinates/tests/test_icrs_observed_transformations.py::test_icrs_consistency[...]
FAILED ../../.tox/py39-test/lib/python3.9/site-packages/astropy/coordinates/tests/accuracy/test_altaz_icrs.py::test_against_pyephem
=== 201 failed, 13912 passed, 1061 skipped, 104 xfailed in 236.68s (0:03:56) ===

cc @mhvk and @adrn

Activity

  1. pllim commented on Feb 8, 2021

    @pllim
    MemberAuthor

    Looking at numpy/numpy@v1.20.0...v1.20.1 , the most suspicious one is numpy/numpy#18337 but I am not sure how that is related to astropy unit comparison...

  2. pllim commented on Feb 8, 2021

    @pllim
    MemberAuthor

    Aha... @saimn tracked down the real reason at #11305.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions