Test failure with coordinates: 'NoneType' object has no attribute 'to_geodetic'

Revert TST: Temporarily xfail test_skyoffset until to_geodetic error is handled properly #11319

There is a test failure in coordinates in the 32bits job:
'NoneType' object has no attribute 'to_geodetic'
https://github.com/astropy/astropy/pull/11274/checks?check_run_id=1772534200

_______________ test_skyoffset[inradec0-expectedlatlon0-tolsep0] _______________

inradec = <Quantity [45., 45.] deg>, expectedlatlon = <Quantity [0., 0.] deg>
tolsep = <Quantity 0.001 arcsec>, originradec = <Quantity [45., 45.] deg>

    @pytest.mark.parametrize("inradec,expectedlatlon, tolsep", [
        ((45, 45)*u.deg, (0, 0)*u.deg, .001*u.arcsec),
        ((45, 0)*u.deg, (0, -45)*u.deg, .001*u.arcsec),
        ((45, 90)*u.deg, (0, 45)*u.deg, .001*u.arcsec),
        ((46, 45)*u.deg, (1*np.cos(45*u.deg), 0)*u.deg, 16*u.arcsec),
        ])
    def test_skyoffset(inradec, expectedlatlon, tolsep, originradec=(45, 45)*u.deg):
        origin = ICRS(*originradec)
        skyoffset_frame = SkyOffsetFrame(origin=origin)
    
        skycoord = SkyCoord(*inradec, frame=ICRS)
        skycoord_inaf = skycoord.transform_to(skyoffset_frame)
        assert hasattr(skycoord_inaf, 'lon')
        assert hasattr(skycoord_inaf, 'lat')
        expected = SkyCoord(*expectedlatlon, frame=skyoffset_frame)
    
        assert skycoord_inaf.separation(expected) < tolsep
        # Check we can also transform back (regression test for gh-11254).
>       roundtrip = skycoord_inaf.transform_to(ICRS())

../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/tests/test_skyoffset_transformations.py:33: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/sky_coordinate.py:627: in transform_to
    return self.__class__(new_coord, **frame_kwargs)
../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/sky_coordinate.py:323: in __init__
    setattr(self, attr, skycoord_kwargs[attr])
../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/sky_coordinate.py:811: in __setattr__
    frame_transform_graph.frame_attributes[attr].__get__(self)
../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/attributes.py:104: in __get__
    out, converted = self.convert_input(out)
../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/attributes.py:451: in convert_input
    transformedobj = value.transform_to(self._frame)
../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/sky_coordinate.py:619: in transform_to
    new_coord = trans(self.frame, generic_frame)
../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/transformations.py:1422: in __call__
    curr_coord = t(curr_coord, curr_toframe)
../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/transformations.py:1025: in __call__
    return supcall(fromcoord, toframe)
../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/builtin_frames/icrs_observed_transforms.py:29: in icrs_to_altaz
    astrom = erfa_astrom.get().apco(altaz_frame)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

frame_or_coord = <AltAz Frame (obstime=None, location=None, pressure=0.0 hPa, temperature=0.0 deg_C, relative_humidity=0.0, obswl=1.0 micron)>

    @staticmethod
    def apco(frame_or_coord):
        '''
        Wrapper for ``erfa.apco``, used in conversions AltAz <-> ICRS and CIRS <-> ICRS
    
        Arguments
        ---------
        frame_or_coord: ``astropy.coordinates.BaseCoordinateFrame`` or ``astropy.coordinates.SkyCoord``
            Frame or coordinate instance in the corresponding frame
            for which to calculate the calculate the astrom values.
            For this function, an AltAz or CIRS frame is expected.
        '''
>       lon, lat, height = frame_or_coord.location.to_geodetic('WGS84')
E       AttributeError: 'NoneType' object has no attribute 'to_geodetic'

../../.tox/py38-test-double/lib/python3.8/site-packages/astropy/coordinates/erfa_astrom.py:45: AttributeError

@mhvk

That's weird. It had been passing on master and I don't see any new erfa or pyerfa release. Any idea, @mhvk or @adrn ?

It passed on master after the merge, and on most of the builds I have looked at, but sometimes it's failing. Another example: https://github.com/astropy/astropy/runs/1754950817?check_suite_focus=true
Probably a transient with a remote service, but it's weird that it happens each time on the 32bits job ?

Keep in mind that the 32-bit job also runs parallel and "run the suite twice" jobs, and it is failing in py38-test-double, so probably some state that is kept dirty after the first run, but not always?

@embray

Hmm, I'd guess this might be related to #11255. There definitely was something tricky going on with frame attributes for SkyOffsetFrame, and maybe I messed up in a way that neither I nor @embray noticed. I do vaguely recall getting similar error messages while working on that...

I can reproduce locally but only if I do this. It doesn't show when I run pytest <filename> twice on the terminal.

python -c "import astropy; astropy.test(test_path='astropy/coordinates/tests/test_skyoffset_transformations.py'); astropy.test(test_path='astropy/coordinates/tests/test_skyoffset_transformations.py')"

So maybe the test runner is "polluting" the test somehow.

Nice sleuthing! As noted in the PR, I'll try to have a quick check if there isn't a real problem underlying this failure... A possibility might be that the coordinate relative to which the SkyOffsetFrame is defined is deleted, but the frame still hangs around, which would not be right.

Yikes, this is tricky: I cannot consistently reproduce getting the error on the 2nd run with your one-liner calling astropy.test twice. The first time I tried, it worked, but now tried a few times again and both calls just pass. Well, at least perhaps that is what happened to your cache clearing too?!

Well, at least it is consistent with not always failing on CI...

Ha, got another failure! But only once every 10 calls or so.

Looking at the trace, I now notice that somehow it is trying to get a location, because it thinks it is dealing with some sort of AltAz frame, even though in reality it is all in ICRS.

Hah, at least some progress: if I move the AltAz tests before the ICRS tests, the first trial of the test file much more consistently gives errors (not always, but mostly; and any subsequent trial is fine); and now pytest also regularly fails.

So, it is not due to the test runner at least, but something semi-persistent, likely not being cleaned up fast enough.

So, at least it is clear where the problem lies:

astropy/astropy/coordinates/sky_coordinate.py

Lines 810 to 811 in 63f90ef

    
           # Validate it 
        
           frame_transform_graph.frame_attributes[attr].__get__(self)

Here, when transforming back from the offset frame to the base frame, the 'origin' attribute is stored on the SkyCoord (because ICRS does not have 'origin'). But the validation is done by the origin attribute of whatever SkyOffsetFrame happens to be last in a set of all frames in the frame transform graph, which, if multiple sky offset frames have been defined, may be an altaz coordinate attribute (somewhat randomly, just if the set happens to be like that). If so, for an ICRS sky offset, it tries to convert the icrs origin to altaz (which is impossible, because it does not have a location).

Now how to solve this is less obvious... I tried simply disabling input verification here, but that gives lots of errors. Perhaps those SkyOffsetFrame attributes should simply never be kept... Though I think the main problem is having an attribute for which the behaviour depends on some internal state.

@mhvk

Wow, even better sleuthing, @mhvk ! Sorry to send you down the wrong rabbit hole.

Is this an acceptable workaround?

Catch error at L810 that you mentioned above and re-raise with a clearer error message.
In the affected test, if we see that error and error message combo, mark it as xfail.

The problem is that the error is actually wrong: it is using the origin instance for a different OffsetFrame than the one that is passed in. It could happen in user code too (if they have offset frames in different coordinate systems).

p.s. I do think we could have a relatively hacky work-around, but need to have some time to think it through.

TST: Temporarily xfail until astropy#11277 is solved

It could happen in user code too (if they have offset frames in different coordinate systems).

I have indeed just run into this while trying to convert AltAz offsets to ICRS offsets. Any suggestions for a workaround, or is trying to use offset frames from different coordinates systems currently unworkable?

In case having a MWE will help with testing, here's what I was preparing for a bug report before I discovered this bug. It fails most of the time:

#!/usr/bin/env python3

import astropy.units as u
from astropy.coordinates import SkyCoord, SkyOffsetFrame, EarthLocation, AltAz, ICRS
from astropy.time import Time


obstime = Time('2021-04-22T13:00:00Z')
location = EarthLocation.from_geodetic(18 * u.deg, lat=-30 * u.deg, height=100 * u.m)
altaz_frame = AltAz(obstime=obstime, location=location)
target = SkyCoord(alt=70 * u.deg, az=150 * u.deg, frame=altaz_frame)

dirs_altaz_offset = SkyCoord(
    lon=[-0.02, 0.01, 0.0, 0.0, 0.0] * u.rad,
    lat=[0.0, 0.2, 0.0, -0.3, 0.1] * u.rad,
    frame=target.skyoffset_frame()
)
dirs_altaz = dirs_altaz_offset.transform_to(altaz_frame)
dirs_icrs = dirs_altaz.transform_to(ICRS())
target_icrs = target.transform_to(ICRS())
dirs_icrs_offset = dirs_icrs.transform_to(target_icrs.skyoffset_frame())
print(dirs_icrs_offset)

Hm, totally confused about how to solve this, but as a workaround for your use case, would it work to replace the last line with

dirs_icrs_offset = dirs_icrs.frame.transform_to(target_icrs.skyoffset_frame())

?

as a workaround for your use case, would it work to replace the last line with

Thanks, that does appear to work (although it means I end up with a SkyOffsetICRS instead of a SkyCoord).

In that case

dirs_icrs_offset = SkyCoord(dirs_icrs.frame.transform_to(target_icrs.skyoffset_frame()))

😄

Thanks, that does seem to work reliably (slightly to my surprise, but I don't really understand the cause of the bug).

Simplify workaround for astropy bug

@adrn

In the interest of providing more examples, I also just ran into this. Here's the code:

import datetime
from pytz import timezone

from IPython import embed

import numpy
from matplotlib import pyplot

from astropy import coordinates
from astropy import units
from astropy.time import Time

def main():

    # Telescope site
    keck = coordinates.EarthLocation.of_site('keck')
    tz = timezone(keck.info.meta['timezone'])

    # Tonight at midnight local time
    tomorrow = datetime.date.today() + datetime.timedelta(days=1)
    midnight = datetime.datetime(tomorrow.year, tomorrow.month, tomorrow.day, 0, 0, 0)
    # Midnight local time converted to UTC (includes daylight savings
    # time offsets)
    midnight -= tz.utcoffset(midnight)
    # Cast to astropy time object with location
    midnight = Time(midnight, location=keck)

    # Pick an object on the meridian at midnight at modest zenith angle
    ra = midnight.sidereal_time('apparent').to('deg')
    dec = keck.lat - 10 * units.deg
    sky_center = coordinates.SkyCoord(ra, dec)

    # Create a cross-hair of coordinates using offsets centered on the
    # above coordinate that span Keck II's FOV
    xoff = numpy.array([0., 0., -10., -5., 0., 0., 10., 5.])/60
    yoff = numpy.array([-10., -5., 0., 0., 10., 5., 0., 0.])/60
    roff = numpy.amax(numpy.absolute(numpy.column_stack((xoff,yoff))), axis=1)
    offset_cross = coordinates.SkyCoord(xoff*units.deg, yoff*units.deg,
                                        frame=sky_center.skyoffset_frame())
    # Works consistently, but always issues a warning
#    sky_cross = coordinates.SkyCoord(offset_cross.frame.transform_to(coordinates.ICRS))
    # Leads to failures more times than not
    sky_cross = offset_cross.transform_to(coordinates.ICRS)
    # Leads to success more times than not
#    sky_cross = offset_cross.transform_to(coordinates.ICRS())

    # Convert coordinates to altitude and azimuth at Keck at midnight
    keck_frame = coordinates.AltAz(obstime=midnight, location=keck)
    field_cross = sky_cross.transform_to(keck_frame)
    field_center = sky_center.transform_to(keck_frame)

    # Get the altitude and azimuth offsets
    field_offset = field_cross.transform_to(coordinates.SkyOffsetFrame(origin=field_center))

    # Repeat but include the atmospheric distortion at 5500 angstrom
    keck_atm = {'pressure': 0.615*units.bar, 'temperature': 2*units.Celsius,
                'relative_humidity': 0.15, 'obswl': 5500*units.angstrom}
    keck_frame_diffraction = coordinates.AltAz(obstime=midnight, location=keck, **keck_atm)

    # When it fails, it always fails on this line.  It never fails on
    # the line above that sets `field_cross`.
    field_cross_diffraction = sky_cross.transform_to(keck_frame_diffraction)
    field_center_diffraction = sky_center.transform_to(keck_frame_diffraction)
    field_offset_diffraction = field_cross.transform_to(coordinates.SkyOffsetFrame(origin=field_center_diffraction))


if __name__ == '__main__':
    main()

As noted in the comments, this line suggested by @adrn:

sky_cross = coordinates.SkyCoord(offset_cross.frame.transform_to(coordinates.ICRS))

seems to work consistently. But it always issues this warning:

WARNING: AstropyDeprecationWarning: Transforming a frame instance to a frame class (as opposed to another frame instance) will not be supported in the future.  Either explicitly instantiate the target frame, or first convert the source frame instance to a `astropy.coordinates.SkyCoord` and use its `transform_to()` method. [astropy.coordinates.baseframe]

The other two lines:

sky_cross = offset_cross.transform_to(coordinates.ICRS)

and

sky_cross = offset_cross.transform_to(coordinates.ICRS())

fail with differing regularity, but the failure always occurs at the line:

field_cross_diffraction = sky_cross.transform_to(keck_frame_diffraction)

Based on the Traceback below, is there some instability in which object is used to specify location, or how location is pass/copied between objects? In the code above, e.g., keck_frame consistently has a defined location, but sky_cross does not.

Traceback (most recent call last):
  File "/Users/westfall/Work/FOBOS/pointing_optimization/bug_example.py", line 73, in <module>
    main()
  File "/Users/westfall/Work/FOBOS/pointing_optimization/bug_example.py", line 67, in main
    field_cross_diffraction = sky_cross.transform_to(keck_frame_diffraction)
  File "/Users/westfall/.virtualenvs/synospec/lib/python3.9/site-packages/astropy/coordinates/sky_coordinate.py", line 627, in transform_to
    return self.__class__(new_coord, **frame_kwargs)
  File "/Users/westfall/.virtualenvs/synospec/lib/python3.9/site-packages/astropy/coordinates/sky_coordinate.py", line 323, in __init__
    setattr(self, attr, skycoord_kwargs[attr])
  File "/Users/westfall/.virtualenvs/synospec/lib/python3.9/site-packages/astropy/coordinates/sky_coordinate.py", line 811, in __setattr__
    frame_transform_graph.frame_attributes[attr].__get__(self)
  File "/Users/westfall/.virtualenvs/synospec/lib/python3.9/site-packages/astropy/coordinates/attributes.py", line 103, in __get__
    out, converted = self.convert_input(out)
  File "/Users/westfall/.virtualenvs/synospec/lib/python3.9/site-packages/astropy/coordinates/attributes.py", line 451, in convert_input
    transformedobj = value.transform_to(self._frame)
  File "/Users/westfall/.virtualenvs/synospec/lib/python3.9/site-packages/astropy/coordinates/sky_coordinate.py", line 619, in transform_to
    new_coord = trans(self.frame, generic_frame)
  File "/Users/westfall/.virtualenvs/synospec/lib/python3.9/site-packages/astropy/coordinates/transformations.py", line 1423, in __call__
    curr_coord = t(curr_coord, curr_toframe)
  File "/Users/westfall/.virtualenvs/synospec/lib/python3.9/site-packages/astropy/coordinates/transformations.py", line 1026, in __call__
    return supcall(fromcoord, toframe)
  File "/Users/westfall/.virtualenvs/synospec/lib/python3.9/site-packages/astropy/coordinates/builtin_frames/cirs_observed_transforms.py", line 52, in cirs_to_altaz
    astrom = erfa_astrom.get().apio13(altaz_frame)
  File "/Users/westfall/.virtualenvs/synospec/lib/python3.9/site-packages/astropy/coordinates/erfa_astrom.py", line 83, in apio13
    lon, lat, height = frame_or_coord.location.to_geodetic('WGS84')
AttributeError: 'NoneType' object has no attribute 'to_geodetic'

Thanks for any help on this!

@bmerry

OK, I think I have a fix in #11730.

@bmerry - thanks very much for your minimal example - it is now part of the test suite! And it was especially helpful as it turns out it actually had a slightly different cause, more properly a bug and with a clean fix (unlike the special-casing I ended up having to do to get rid of the test failures that started this).

Navigation Menu

Repository navigation

Test failure with coordinates: 'NoneType' object has no attribute 'to_geodetic' #11277

3 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	# Validate it
	frame_transform_graph.frame_attributes[attr].__get__(self)

Test failure with coordinates: 'NoneType' object has no attribute 'to_geodetic' #11277

Description

Activity

pllim commented on Jan 27, 2021

saimn commented on Jan 27, 2021

pllim commented on Jan 27, 2021

mhvk commented on Jan 28, 2021

pllim commented on Feb 1, 2021

mhvk commented on Feb 1, 2021

mhvk commented on Feb 2, 2021

mhvk commented on Feb 2, 2021

mhvk commented on Feb 2, 2021

mhvk commented on Feb 2, 2021

pllim commented on Feb 2, 2021

mhvk commented on Feb 2, 2021

3 remaining items

bmerry commented on Apr 23, 2021

bmerry commented on Apr 23, 2021

adrn commented on Apr 23, 2021

bmerry commented on Apr 26, 2021

adrn commented on Apr 26, 2021

bmerry commented on Apr 27, 2021

kbwestfall commented on May 7, 2021

mhvk commented on May 11, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions