Skip to content

Make coordinates.solar_system_ephemeris accept local ephemeris file - #8767

Merged
mhvk merged 4 commits into
astropy:masterfrom
rhandberg:master
May 31, 2019
Merged

mhvk merged 4 commits into
astropy:masterfrom
rhandberg:master

Conversation

@rhandberg

Copy link
Copy Markdown
Contributor

Make coordinates.solar_system_ephemeris accept local ephemeris file as input.

New dependency of os.path is needed, because tests revealed that standard Windows paths would pass through urlparse without throwing an error. I therefore switched to use os.path.isfile to test if the input was a file path.

Fixes #8750

@pllim pllim added this to the v4.0 milestone May 28, 2019
@pllim

pllim commented May 28, 2019

Copy link
Copy Markdown
Member

Thanks! Need to add test(s) and a change log.

@pllim
pllim requested review from adrn and eteq May 28, 2019 15:04
@codecov

codecov Bot commented May 28, 2019

Copy link
Copy Markdown

Codecov Report

Merging #8767 into master will decrease coverage by <.01%.
The diff coverage is 12.5%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #8767      +/-   ##
==========================================
- Coverage   86.96%   86.95%   -0.01%     
==========================================
  Files         400      400              
  Lines       59448    59451       +3     
  Branches     1100     1100              
==========================================
+ Hits        51697    51698       +1     
- Misses       7110     7112       +2     
  Partials      641      641
Impacted Files Coverage Δ
astropy/coordinates/solar_system.py 69.37% <12.5%> (-0.69%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 660d73b...f8045b3. Read the comment docs.

@codecov

codecov Bot commented May 28, 2019 •

Copy link
Copy Markdown

Codecov Report

Merging #8767 into master will increase coverage by 0.01%.
The diff coverage is 71.42%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #8767      +/-   ##
==========================================
+ Coverage   86.96%   86.97%   +0.01%     
==========================================
  Files         400      400              
  Lines       59454    59456       +2     
  Branches     1100     1100              
==========================================
+ Hits        51703    51714      +11     
+ Misses       7110     7101       -9     
  Partials      641      641
Impacted Files Coverage Δ
astropy/coordinates/solar_system.py 75% <71.42%> (+4.93%) ⬆️
astropy/units/function/__init__.py 100% <0%> (ø) ⬆️
astropy/coordinates/builtin_frames/__init__.py 100% <0%> (ø) ⬆️
astropy/samp/hub.py 77.52% <0%> (+0.13%) ⬆️
astropy/utils/data.py 81.15% <0%> (+0.22%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1658b5a...6c039c0. Read the comment docs.

@pllim
pllim requested a review from mhvk May 28, 2019 15:10
@pllim

pllim commented May 28, 2019

Copy link
Copy Markdown
Member

Also before merge, the commits need to be squashed to get rid of the "merge commit" message. Please do not use git pull, but instead use git fetch followed by git rebase. Thanks!

@rhandberg

Copy link
Copy Markdown
Contributor Author

Also before merge, the commits need to be squashed to get rid of the "merge commit" message. Please do not use git pull, but instead use git fetch followed by git rebase. Thanks!

I must admit I have no idea about how to do this... Running those two commands doesn't seem to change anything...???

@pllim

pllim commented May 28, 2019

Copy link
Copy Markdown
Member

@mhvk

mhvk commented May 28, 2019

Copy link
Copy Markdown
Contributor

@rhandberg - once you have done the merge commit, the rebase keeps it in place, but one could get rid of it by doing git rebase -i ... and then explicitly removing it. It is no big deal, though!

@mhvk

mhvk commented May 28, 2019

Copy link
Copy Markdown
Contributor

@rhandberg - the actual PR looks good! But we do need a test! (in coordinates/tests/test_solar_system.py) - this is why codecov complains...

Ideally, the test would use a tiny ephemeris file, but I don't know if such a thing exists (or is easy to create). Alternatively, since we download de432s anyway for the tests, perhaps there is a way to get the path to where it is downloaded (@pllim, would you know how?) and pass that in. If you then also add a test that tries to open just some random astropy file, you can check that with your PR this gives an error from SPK (rather than a URL parse error).

Easier, but no less importantly, could you add an entry in CHANGES.rst under "New Features" for astropy.coordinates? That will make astro-bot happy.

I wondered about adding something to the general documentation, but think that is not necessary: people should just look at the docstring you already changed. That said, for reasons I'm not sure about, docs/coordinates/solarsystem.rst doesn't properly link to solar_system_ephemeris - feel free to fix that! (but you don't have to.)

@pllim

pllim commented May 29, 2019 •

Copy link
Copy Markdown
Member

get the path

Would astropy.utils.data.get_cached_urls help? Or theoretically astropy.utils.data.download_file will grab the cached version, if available.

if value.lower() in ('de430', 'de432s'):
value = ('http://naif.jpl.nasa.gov/pub/naif/generic_kernels'
'/spk/planets/{:s}.bsp'.format(value.lower()))

return SPK.open(download_file(value, cache=True))

So, to clarify further, theoretically, you could do this (unless pytest does weird things to caches):

>>> from astropy.utils.data import download_file, get_cached_urls
>>> # The URL is just example, replace with the URL you desire
>>> get_cached_urls()
['http://data.astropy.org/l1448/l1448_13co.fits', ...]
>>> # No actual download would happen in this line below
>>> file_path = download_file('http://data.astropy.org/l1448/l1448_13co.fits', cache=True)
>>> file_path
'C:\\Users\\...\\.astropy\\cache\\download\\py3\\somehashvalue'

Have to use os.path.isfile, because urlparse does not fail on Windows paths
@rhandberg

Copy link
Copy Markdown
Contributor Author

What do you think of the following test? It should test the loading of ephemeris using URLs and file paths and that they give the same result.

@pytest.mark.remote_data
@pytest.mark.skipif('not HAS_JPLEPHEM')
@pytest.mark.parametrize('time', (Time('1960-01-12 00:00'),
                                  Time('1980-03-25 00:00'),
                                  Time('2010-10-13 00:00')))
def test_url_or_file_ephemeris(time):
	# URL for ephemeris de432s used for testing:
    url = 'http://naif.jpl.nasa.gov/pub/naif/generic_kernels/spk/planets/de432s.bsp'

	# Pass the ephemeris directly as a URL.
    coord_by_url = get_body('earth', time, ephemeris=url)

    # Translate the URL to the cached location on the filesystem.
    # Since we just used the url above, it should already have been downloaded.
    filepath = download_file(url, cache=True)

	# Get the coordinates using the file path directly:
    coord_by_filepath = get_body('earth', time, ephemeris=filepath)

	# Using the URL or filepath should give exactly the same results:
    assert_quantity_allclose(coord_by_url.ra, coord_by_filepath.ra)
    assert_quantity_allclose(coord_by_url.dec, coord_by_filepath.dec)
    assert_quantity_allclose(coord_by_url.distance, coord_by_filepath.distance)

Maybe a separate test is needed to check that it throws and error if you provide a non-existing file or URL?

@mhvk

mhvk commented May 29, 2019

Copy link
Copy Markdown
Contributor

@pllim - thanks, that's just the thing! @rhandberg - test looks excellent! And agreed that a test with a non-existing file would be good to add - given your code, this will also check the bad url path (which of course we know works from your initial issue!). Maybe also a test with an existing file that is not an ephemeris file.

@rhandberg

Copy link
Copy Markdown
Contributor Author

Great! What do you think about this for the second test where we are testing for non-existing URL and file path as well as a file that is not an ephemeris file... Actually I'm just passing the current Python file...

@pytest.mark.remote_data
@pytest.mark.skipif('not HAS_JPLEPHEM')
def test_url_or_file_ephemeris_wrong_input():

	time = Time('1960-01-12 00:00')

	# Try loading a non-existing URL:
	with pytest.raises(HTTPError):
		get_body('earth', time, ephemeris='http://data.astropy.org/path/to/nonexisting/file.bsp')

	# Try loading a non-existing file:
	with pytest.raises(ValueError):
		get_body('earth', time, ephemeris='/path/to/nonexisting/file.bsp')

	# Try loading a file that does exist, but is not an ephemeris file:
	with pytest.raises(ValueError):
		get_body('earth', time, ephemeris=__file__)

@pllim

pllim commented May 29, 2019

Copy link
Copy Markdown
Member

FWIW

	# Try loading a non-existing URL:
	with pytest.raises(HTTPError):
		get_body('earth', time, ephemeris='http://data.astropy.org/path/to/nonexisting/file.bsp')

can be in its own test function decorated with remote_data. That way, the other two tests can run even when remote_data is not specified (and not needed).

@rhandberg

Copy link
Copy Markdown
Contributor Author

@pllim - Very good point! I will split that test into two.

@rhandberg

Copy link
Copy Markdown
Contributor Author

Tests have now been added, but codecov seems to still fail... Looks like it is because the whole solar_system.py file (not just from this PR) is not covered as well as the rest of astropy

@mhvk mhvk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks all OK modulo two small items.

Comment thread CHANGES.rst Outdated
^^^^^^^^^^^^^^^^^^^

- Changed ``coordinates.solar_system_ephemeris`` to also accept local files
as input. The ephemeris can not be selected by either keyword (e.g. 'jpl',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can not -> can now

Comment thread astropy/coordinates/solar_system.py Outdated
value = DEFAULT_JPL_EPHEMERIS

if value.lower() in ('de430', 'de432s'):
elif value.lower() in ('de430', 'de432s'):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should remain an if - since DEFAULT_JPL_EPHEMERIS = 'de430'

I guess I never added a test since I didn't want to download an enormous file for every test run...

@rhandberg

Copy link
Copy Markdown
Contributor Author

@mhvk - Good catches! Those two issues have been fixed now.

@mhvk mhvk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mhvk
mhvk merged commit 17ef367 into astropy:master May 31, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make coordinates.solar_system_ephemeris accept local ephemeris file

3 participants