Repository navigation
Assigning new data to an existing HDU won't update scaling consistently #5559
Description
When creating a brand new ImageHDU, this:
im = ImageHDU(data=data, header=existing_header)
will deal properly with the header scaling, adding or subtracting BZERO/BSCALE to fit the data type, no matter what the data type is, or whether the original header has scaling information on it or not. If we set the data after creation, the behavior is not consistent. Let's show this.
>>> uint8 = np.array([[1,2,3,4],[4,3,2,1]], dtype='uint8')
>>> uint64 = np.array([[1,2,3,4],[4,3,2,1]], dtype='uint64')
Now, let's create a new image with a data type that doesn't require scaling, and replace the data with the unsigned 64 bit array:
>>> im = ImageHDU(uint8)
>>> im.header
XTENSION= 'IMAGE ' / Image extension
BITPIX = 8 / array data type
NAXIS = 2 / number of array dimensions
NAXIS1 = 4
NAXIS2 = 2
PCOUNT = 0 / number of parameters
GCOUNT = 1 / number of groups
>>> im.data = uint64
>>> im.header
XTENSION= 'IMAGE ' / Image extension
BITPIX = 64 / array data type
NAXIS = 2 / number of array dimensions
NAXIS1 = 4
NAXIS2 = 2
PCOUNT = 0 / number of parameters
GCOUNT = 1 / number of groups
BSCALE = 1
BZERO = 9223372036854775808
It behaves as expected. Now:
>>> im = ImageHDU(uint64)
>>> im.header
XTENSION= 'IMAGE ' / Image extension
BITPIX = 64 / array data type
NAXIS = 2 / number of array dimensions
NAXIS1 = 4
NAXIS2 = 2
PCOUNT = 0 / number of parameters
GCOUNT = 1 / number of groups
BSCALE = 1
BZERO = 9223372036854775808
>>> im.data = uint8
>>> im.header
XTENSION= 'IMAGE ' / Image extension
BITPIX = 8 / array data type
NAXIS = 2 / number of array dimensions
NAXIS1 = 4
NAXIS2 = 2
PCOUNT = 0 / number of parameters
GCOUNT = 1 / number of groups
BSCALE = 1
BZERO = 9223372036854775808
As we can see, scaling info is preserved, when it shouldn't. This works properly if the new array is of 16/32/64 integer (signed or unsigned), but fails with 8 bit and floating point data.
Activity
MSeifert04 commentedon Dec 5, 2016
Thanks for reporting this issue.
I have one question: bscale and bzero are only needed when reading a file - Are these values preserved or updated when you write your
imto a file and read it back?Also there have been several edits to bscale and bzero, which version of astropy are you using?
rcardenes commentedon Dec 6, 2016
We're writing this back to disk, and the headers are preserved when writing, so when reading the data is screwed up. In principle I only discovered this because for some reason I can't really remember (sigh), I needed to create an image like this:
I'm not doing this any longer, so it shouldn't be an issue for us, but I still think it was worth reporting.
I've observed the bug both in AstroPy 1.2.1 and in 1.3.dev17297
saimn commentedon Mar 21, 2019
It looks like this case was excluded on purpose in #5053, in particular #5053 (comment): do you remember why @mwcraig ?
From what I can understand it seems that the goal is to be able to know that the original data was unsigned, once it has been scaled.
If I modify the code to handle the case above, allowing the removal of BSCALE/BZERO when the new data is unsigned, then the test added in #5053 fails, as well as
test_uint_header_consistencyfrom #2305.