Skip to content

Conversation

@mhvk
Copy link
Contributor

@mhvk mhvk commented Oct 28, 2013

To ensure UT1 can be calculated for dates up to ~Oct'13. Obviously, we'll need to automate this at some point...

@eteq
Copy link
Member

eteq commented Oct 28, 2013

Ah, good idea! Assuming this passes it seems fine.

@mhvk
Copy link
Contributor Author

mhvk commented Oct 29, 2013

travis passed (as it should!) -- this should be OK.

@astrofrog
Copy link
Member

Does it make sense to store it compressed? This will cause the whole file to be added to the repo every time it is changed, right? (500k each time?) What is it uncompressed?

@astrofrog
Copy link
Member

(@eteq @mhvk - I'm starting to mark these new issues as 0.3.1 as they are not release critical, but of course if they get merged this week then they make the cut)

@mhvk
Copy link
Contributor Author

mhvk commented Oct 29, 2013

This should be 0.3 -- this is a file that should always be updated whenever a release happens (which I why I mentioned we need to automate it -- now made new issue about that).

The iers file is 3 MB uncompressed, so 6 times larger, but of course only little bits of text get added over a year (since it covers everything since 1962, it increases by about 2% each year or ~100kB/yr). I now see your point that if we do this regularly, the developers at least soon loose any benefits we would have from compression, especially since git does store differences compressed. So, I changed it to uncompressed.

@eteq
Copy link
Member

eteq commented Oct 29, 2013

@astrofrog @mhvk - I see the point of having git track the changes, and it may actually be more efficient anyway because of git's built-in compression. But our guidelines have said over a megabyte is too big... Remind me again if this is a temporary measure, @mhvk? That is, will we permanently keep this file around, or is it a placeholder until we get a better online download working?

Also, @mhvk, can you rebase this to eliminate the commit with the gzipped version? Otherwise we needlessly store a 500k binary file in git that never actually gets used.

@eteq
Copy link
Member

eteq commented Oct 29, 2013

Also, @mhvk - it looks like the error is due to a bug in astropy.utils.data that I will fix shortly... but it also means that in its current form, this is trying to download the IERS file... And presumably it shouldn't be doing that. So perhaps you can add a test to check if it downloads or not (you might do this in a separate PR if you don't think you can get to it by Friday, as this is more important than that is)?

@mhvk
Copy link
Contributor Author

mhvk commented Oct 29, 2013

@eteq - OK, rebased. Thanks also for pointing to the download issue -- I had to change the filename in the main iers file.

The consensus from when IERS was introduced was that we should provide an up-to-date IERS file in the package, so that one can get UT1 for any time before the packaging time. The alternative would be that upon installation it downloads these files (or upon first use of UT1), which I think was considered too cumbersome and prone to breakage. We may want to revisit this... (I'll add it to #1697)

@eteq
Copy link
Member

eteq commented Oct 29, 2013

Ah, right, ok. So we likely will have to change this in some form at every release. Then it definitely makes sense to use the uncompressed version.

eteq added a commit to eteq/astropy that referenced this pull request Oct 29, 2013
@mhvk
Copy link
Contributor Author

mhvk commented Oct 29, 2013

@eteq - as you said, this is causing an error with get_pkg_data_filename -- how did that get past travis? Let me know if I need to rebase on something.

@eteq
Copy link
Member

eteq commented Oct 29, 2013

@mhvk - I think the underlying issue is that we don't run the remote-data tests in travis (#734)...

I just put a commit in master to fix this because it's clearly a bug, so you can try rebasing against that. However, I think the fact that you reach that part of get_pkg_data_filename at all means it still isn't finding the local copy (and it will fail anyway, because the remote file doesn't exist). It might be easier for you to try to diagnose this locally...

@mhvk
Copy link
Contributor Author

mhvk commented Oct 29, 2013

@eteq - there is something truly odd, perhaps related to the filename, in that os.path.isfile(datafn) returns False while trying to make configuration items (but not in an interactive ipython session). I'm still investigating, but any hints appreciated (don't really want to rename the file).

@mhvk
Copy link
Contributor Author

mhvk commented Oct 29, 2013

@eteq - never mind - the problem is that the file does not actually get copied in building.

@astrofrog
Copy link
Member

Ah, you need to define it in setup_package as data to be included in the sub-package. Check out other sub-packages for examples of this.

@astrofrog
Copy link
Member

e.g. astropy/table/setup_package.py has

def get_package_data():
    paths = [os.path.join('data', '*.js'),
             os.path.join('data', '*.css'),
             ]
    return {'astropy.table': paths} 

@mhvk
Copy link
Contributor Author

mhvk commented Oct 29, 2013

@astrofrog, @eteq - thanks! I must have once done this (or someone else did it for me when implementing IERS), but I had clearly forgotten. It now builds OK.... (famous last words).

@mhvk
Copy link
Contributor Author

mhvk commented Oct 29, 2013

Problem now seems just to be a very busy travis (timeouts in building)...

@eteq
Copy link
Member

eteq commented Oct 30, 2013

Looks like everything's passing, so I'll just go ahead and merge. Thanks @mhvk

eteq added a commit that referenced this pull request Oct 30, 2013
Update IERS eopc04_IAU2000.62-now file for 0.3 release
@eteq eteq merged commit 57f0683 into astropy:master Oct 30, 2013
@mhvk mhvk deleted the time-update-iers-table branch October 30, 2013 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants