Skip to content

Fix uint conversion after slicing - #8982

Merged
saimn merged 1 commit into
astropy:masterfrom
saimn:fits-fix-uint-fitsrec
Jul 12, 2019
Merged

saimn merged 1 commit into
astropy:masterfrom
saimn:fits-fix-uint-fitsrec

Conversation

@saimn

@saimn saimn commented Jul 11, 2019

Copy link
Copy Markdown
Contributor

Fix #5490.

The _uint attribute was not transfered to the sliced array, so the data
was later converted either correctly (if it was converted first before
slicing) or as float in the other case.

Fix astropy#5490.

The _uint attribute was not transfered to the sliced array, so the data
was later converted either correctly (if it was converted first before
slicing) or as float in the other case.
@saimn saimn added the io.fits label Jul 11, 2019
@saimn saimn added this to the v2.0.15 milestone Jul 11, 2019
@saimn
saimn requested review from MSeifert04 and mhvk July 11, 2019 20:27
@pllim pllim added the Bug label Jul 11, 2019

@mhvk mhvk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thanks for figuring out this very old issue!

Since my comments are really just queries, approving now.


# We got a view; change it back to our class, and add stuff
out = out.view(type(self))
out._uint = self._uint

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vaguely recall adding some of the above code, but looking at it again I wonder a bit why... Normally, an ndarray subclass is supposed to do this in __array_finalize__, but here of course the view as a recarray and then the view back here circumvent that (and from a quick look, just doing out.__arrray_finalize__(self) would seem unlikely to do quite the right thing)

Anyway, I think the fix here is good, though perhaps have a quick look at __array_finalize__ and see if anything else is missing (since _uint at least is updated there).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I should have mentioned that I requested your review because you changed a bit this code in the past (e.g. #6955), but the issue was already there before. The problem is that with the recarray view we loose the FITS_rec attributes, so they are no more available in __array_finalize__ when viewing back to FITS_rec.
I'm not completely sure about the other attributes, I think that the missing ones can be recomputed from the coldefs.

assert_array_equal(data[sel]['b'], dataref[sel])
assert data[sel]['b'].dtype == dataref[sel].dtype

with fits.open(self.temp('test_scaled_slicing.fits')) as hdulist:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we opening the file twice to prevent being fooled by a cache or so?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes somewhat, in the first case as data['b'] is accessed the data is converted and cached, so before the fix data[sel] had the wrong value for _uint but was still able to print correct values because the cache is copied.

@saimn
saimn merged commit c25a012 into astropy:master Jul 12, 2019
@saimn
saimn deleted the fits-fix-uint-fitsrec branch July 12, 2019 14:23
@saimn

saimn commented Jul 12, 2019

Copy link
Copy Markdown
Contributor Author

Thanks for the review @mhvk !

@mhvk

mhvk commented Jul 12, 2019

Copy link
Copy Markdown
Contributor

And thanks for the fix, and for explaining - all makes sense. (Though I think we should be able to be more properly ndarray subclassy - but for another time...)

bsipocz pushed a commit that referenced this pull request Jul 15, 2019
bsipocz pushed a commit that referenced this pull request Jul 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Slicing FITS_rec objects produces unexpected dtypes.

3 participants