Repository navigation
fits.CompImageHDU writes unexpected/incorrect BZERO keyword after conversion from unsigned image data #5999
Description
When I read or create a FITS image HDU with unsigned integer data, then convert it to floating point and save it in a CompImageHDU, an additional BZERO keyword & value get written into the relevant header, even with Astropy telling me it's not there.
Here is a minimal example:
from __future__ import print_function
import sys
from tempfile import NamedTemporaryFile
import numpy as np
from astropy.io import fits
data = np.arange(0, 100, dtype=np.uint16)
hdu = fits.ImageHDU(data=data)
data = hdu.data.astype(np.float64)
header = hdu.header
hdulist = [fits.PrimaryHDU(),
fits.ImageHDU(data=data, header=header, name='normal'),
fits.CompImageHDU(data=data, header=header, name='compressed')]
with NamedTemporaryFile() as tmpfile:
name = tmpfile.name
fits.HDUList(hdulist).writeto(name, output_verify='warn')
print('HDU list as in memory')
for hdu in hdulist:
print('BZERO in', hdu.name, ':', hdu.header.get('bzero'))
print('BSCALE in', hdu.name, ':', hdu.header.get('bscale'))
print()
print('Re-opening test file')
with fits.open(name) as hdulist:
for hdu in hdulist:
print('BZERO in', hdu.name, ':', hdu.header.get('bzero'))
print('BSCALE in', hdu.name, ':', hdu.header.get('bscale')) This produces:
HDU list as in memory
BZERO in PRIMARY : None
BSCALE in PRIMARY : None
BZERO in NORMAL : None
BSCALE in NORMAL : None
BZERO in COMPRESSED : None
BSCALE in COMPRESSED : None
Re-opening test file
BZERO in PRIMARY : None
BSCALE in PRIMARY : None
BZERO in NORMAL : None
BSCALE in NORMAL : None
BZERO in COMPRESSED : 32768
BSCALE in COMPRESSED : None
The penultimate output line is obviously in error.
Changing the dtype in np.arange to uint32 shows a value of 2147483648, while using a signed integer or floating point dtype shows the BZERO keyword to be absent.
I find the interesting part that the BZERO key gets written (without an accompanying BSCALE key!), but the in-memory header does not contain it whatsoever. All other header keywords seem to be ok.
I've not had time yet to explore this in more depth in Astropy, nor compare it with other current issues with CompImageHDU to see if they are related.
So far, I've seen this happen with various combinations of Python 2.7 & 3.6, Astropy master, 1.3.0, 1.32 and 1.2.1, on Linux and macOS.
Activity
evertrol commentedon May 3, 2017
Additional note: the header of the "compressed image" actually looks as follows:
Which is clearly an image HDU, not a binary table HDU as I would expect for a compressed image.
Perhaps I'm misusing
CompImageHDUsomehow, but I don't see where.pllim commentedon May 3, 2017
The last PR that touched it was #5053. @mwcraig ?
mwcraig commentedon May 3, 2017
Will look at it by Sunday some time, at latest. Next two days will be busy....
saimn commentedon May 9, 2017
This HDU lies to you :
The issue looks indeed very similar to #5053.
Fix creating CompImageHDU from header with bscale/bzero
Fix creating CompImageHDU from header with bscale/bzero