Skip to content

Implement support for the -TAB algorithm in WCS#9641

Merged
nden merged 1 commit intoastropy:masterfrom
mcara:wrap-tabprm
Jan 8, 2020
Merged

Implement support for the -TAB algorithm in WCS#9641
nden merged 1 commit intoastropy:masterfrom
mcara:wrap-tabprm

Conversation

@mcara
Copy link
Contributor

@mcara mcara commented Nov 21, 2019

This pull request is to implement support for the -TAB algorithm in FITS WCS as described in WCS Paper III.

This PR partially addresses #2362, #5462, #7883 in the sense that astropy.wcs now will be able to support -TAB in WCS if already defined in a HDUList object. However this PR does not provide "write" support for -TAB: the files must be created using external scripts.

@timj
Copy link
Contributor

timj commented Nov 21, 2019

This is great. LSST recently started writing files with -TAB for a small spectrograph we have. I can donate a sample file if you like? Also, wcsLib itself generates a rather complex -TAB example file. Have you tested with that file?

My test files are at: https://jira.lsstcorp.org/browse/DM-21797

@mcara
Copy link
Contributor Author

mcara commented Nov 21, 2019

@timj I got your files (Thanks!) and I was able to create astropy.wcs.WCS objects without issues for both of them. The numbers appear "reasonable" to me though I really have no idea whether the coordinates that I am getting are what you expect to see.

@nden
Copy link
Contributor

nden commented Nov 21, 2019

@timj Could someone on the LSST team test this PR as well?

@nden nden requested a review from MSeifert04 November 21, 2019 17:21
@nden
Copy link
Contributor

nden commented Nov 21, 2019

@MSeifert04 Since you've looked at the C wrappers before do you think you can review this one?

@MSeifert04
Copy link
Contributor

MSeifert04 commented Nov 21, 2019

@mcara Is it feasible to undo the purely style-related changes? It makes the PR quite complicated to review and a bit of information is lost (the /*@null@*/, etc.) and I find some of the changes problematic, e.g. removal of curly braces.

@MSeifert04
Copy link
Contributor

MSeifert04 commented Nov 21, 2019

This sounds like a great addition. Thank you @mcara . I'll do a full review soon, but I'm not sure if I will be able to do the review today/tomorrow.

@mcara
Copy link
Contributor Author

mcara commented Nov 21, 2019

@MSeifert04 Thanks a lot for a thorough review. I hope I have addressed all your comments in the latest commit.

0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
(reprfunc)PyTabprm___str__, /*tp_repr*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for removing the repr?

If that wasn't an accidental removal we probably should also remove the function implementation of PyTabprm___str__

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed __repr__ because I hated the output of it when I was debugging my code. However, it is really a matter of taste and whether people prefer excessive vs succinct output. To understand (=see for yourself) what I am talking about, try:

>>> from astropy import wcs
>>> 5 * [wcs.WCS().wcs]

and now please find the cdelt of the 2nd WCS in the printed output. So, for me it was easier to see how many tabprm or wtbarr structures I had in the members WCS.wcs.tab and WCS.wcs.wtb. If I wanted to see the content of one particular structure, I would do print(WCS.wcs.wtb[0]).

Maybe my above example is not appropriate for a single WCS object (and that is how it is used most of the time), but for lists of tabprm and wtbarr structures, it felt overwhelming. Maybe a more compact format can be found for __repr__ that is also more informative than a just object address.

@mcara
Copy link
Contributor Author

mcara commented Nov 22, 2019

@pllim Could you please be so kind as to help figuring out what is the issue with tests on Windows: https://travis-ci.org/astropy/astropy/jobs/615643939#L1980-L1990 Thank you!

@pllim
Copy link
Member

pllim commented Nov 22, 2019

I can tell you it probably segfaulted but I have no desire to debug it. You can try have the CI install pytest-faulthandler and see if that gives you anything useful. Otherwise, you might need to run it locally on a Windows machine with gdb or something equivalent. Maybe @jhunkeler got better ideas.

@mcara mcara force-pushed the wrap-tabprm branch 3 times, most recently from 3be97d3 to 04835ef Compare January 7, 2020 04:40
@mcara
Copy link
Contributor Author

mcara commented Jan 7, 2020

@MSeifert04 @nden I believe now unit tests are all passing. The reported failures are unrelated to this PR. I found another bug in WCSLIB due to which an element with a very large index ((unsigned int) -1) before the allocated memory block was being accessed. I implemented fix in cextern/wcslib until we get a new release of WCSLIB.

I will report this bug.

@mcara mcara requested a review from MSeifert04 January 7, 2020 05:59
@MSeifert04
Copy link
Contributor

@mcara Great 👍 I'm not sure what our policy is regarding extern libraries but I think we probably need to apply it as "patch" instead of fixing it in the extern code. Just to make it transparent what we applied and what is an extern library.

@nden
Copy link
Contributor

nden commented Jan 8, 2020

I don't think we have a policy about that but in the past we've applied "patches" to a library in cextern. And by "patch" I mean make the change in the same PR which updates wcslib. @MSeifert04 do you mean apply the change to wcslib in a separate PR or something more sophisticated?

@MSeifert04
Copy link
Contributor

do you mean apply the change to wcslib in a separate PR or something more sophisticated?

I am actually not sure how we should patch extern libraries. But I see that my comment was misleading (probably shouldn't comment when tired 😓 ). What I meant was that the commit that fixed wcslib also changed our code-base which I think is problematic, I think the fix for an extern code-base should be restricted to the extern code-base (so that we can easily revert it with git revert or similar) when the new version is out.

If the fix should be in a separate PR or how we should apply it (and what would be possible) is something I don't know.

@mcara
Copy link
Contributor Author

mcara commented Jan 8, 2020

Now WCSLIB fix is in a separate commit.

@nden
Copy link
Contributor

nden commented Jan 8, 2020

@mcara Actually do you mind applying it in a separate PR (to be merged before this one)? That way this PR contains only the -TAB implementation in astropy and the wcslib change is easy to find.

@mcara
Copy link
Contributor Author

mcara commented Jan 8, 2020

@nden Allright. Will do.

@nden
Copy link
Contributor

nden commented Jan 8, 2020

This looks ready to be merged. One last request @mcara, could you add an entry in the "what's new in 4.1" file? There was a recent decision to add entries there with the PR and not before a release.

@MSeifert04
Copy link
Contributor

MSeifert04 commented Jan 8, 2020

Yep, I think this looks okay now.

Thanks @mcara !

@nden
Copy link
Contributor

nden commented Jan 8, 2020

Thanks @mcara, @MSeifert04 and everyone else who contributed!
@MSeifert04 approved in a comment so I'm merging it.

@nden nden merged commit 3fdf53b into astropy:master Jan 8, 2020
@mcara
Copy link
Contributor Author

mcara commented Jan 8, 2020

Thank you @nden and @MSeifert04 for constructive comments and bug finding work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants