fits.open should raise useful exception when it's given a non FITS file #7569

Activity

1. bsipocz commented on Jun 17, 2018

   

   bsipocz

   on Jun 17, 2018

   Last edited by bsipocz

   Member

   More actions

   I got a messed up terminal while trying to reproduce, but after a reset and go into the debugger, rerunning the offending line got this.

   ipdb> file_name
   'Maps/HST/eHst428013.tar'
   

   Then the full traceback when trying to open the file:

   In [20]: a = fits.open('Maps/HST/eHst428013.tar', ignore_missing_end=True)
   In [21]: a
   Out[21]: [<astropy.io.fits.hdu.base._ValidHDU object at 0x115c02940>]
   
   In [22]: a[0].header
   Out[22]: ---------------------------------------------------------------------------
   ValueError                                Traceback (most recent call last)
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/astropy/io/fits/card.py in _fix_value(self)
       855             try:
   --> 856                 value, comment = valuecomment.split('/', 1)
       857                 self.value = value.strip()
   
   ValueError: not enough values to unpack (expected 2, got 1)
   
   During handling of the above exception, another exception occurred:
   
   ValueError                                Traceback (most recent call last)
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/IPython/core/formatters.py in __call__(self, obj)
       700                 type_pprinters=self.type_printers,
       701                 deferred_pprinters=self.deferred_printers)
   --> 702             printer.pretty(obj)
       703             printer.flush()
       704             return stream.getvalue()
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/IPython/lib/pretty.py in pretty(self, obj)
       398                         if cls is not object \
       399                                 and callable(cls.__dict__.get('__repr__')):
   --> 400                             return _repr_pprint(obj, self, cycle)
       401 
       402             return _default_pprint(obj, self, cycle)
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
       693     """A pprint that just redirects to the normal repr function."""
       694     # Find newlines and replace them with p.break_()
   --> 695     output = repr(obj)
       696     for idx,output_line in enumerate(output.splitlines()):
       697         if idx:
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/astropy/io/fits/header.py in __repr__(self)
       248 
       249     def __repr__(self):
   --> 250         return self.tostring(sep='\n', endcard=False, padding=False)
       251 
       252     def __str__(self):
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/astropy/io/fits/header.py in tostring(self, sep, endcard, padding)
       624         lines = []
       625         for card in self._cards:
   --> 626             s = str(card)
       627             # Cards with CONTINUE cards may be longer than 80 chars; so break
       628             # them into multiple lines
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/astropy/io/fits/card.py in __str__(self)
       202 
       203     def __str__(self):
   --> 204         return self.image
       205 
       206     def __len__(self):
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/astropy/io/fits/card.py in image(self)
       503 
       504         if self._image and not self._verified:
   --> 505             self.verify('fix+warn')
       506         if self._image is None or self._modified:
       507             self._image = self._format_image()
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/astropy/io/fits/verify.py in verify(self, option)
        71             return
        72 
   ---> 73         errs = self._verify(opt)
        74 
        75         # Break the verify option into separate options related to reporting of
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/astropy/io/fits/card.py in _verify(self, option)
      1070                          'at column 8).'.format(self.keyword),
      1071                 fix_text=fix_text,
   -> 1072                 fix=self._fix_value))
      1073 
      1074         # verify the key, it is never fixable
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/astropy/io/fits/verify.py in run_option(self, option, err_text, fix_text, fix, fixable)
        45         else:
        46             if fix:
   ---> 47                 fix()
        48             text += '  ' + fix_text
        49 
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/astropy/io/fits/card.py in _fix_value(self)
       858                 self.comment = comment.strip()
       859             except (ValueError, IndexError):
   --> 860                 self.value = valuecomment
       861             self._valuestring = self._value
       862             return
   
   /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/astropy/io/fits/card.py in value(self, value)
       321                     'FITS header values must contain standard printable ASCII '
       322                     'characters; {!r} contains characters not representable in '
   --> 323                     'ASCII or non-printable characters.'.format(value))
       324         elif isinstance(value, bytes):
       325             # Allow str, but only if they can be decoded to ASCII text; note
   
   ValueError: FITS header values must contain standard printable ASCII characters; '202.4695228762\nCRVAL2  = 47.19541750707\nCD1_1   = 2.22562e-05\nCD1_2   =' contains characters not representable in ASCII or non-printable characters.
   

   On the other hand I think it's worth reporting this upstream to the archives as imo they should aim to provide fits files that follows the fits standard. Update: When reading the fits files individually, no warnings are issued, so there is nothing wrong with the archives.
2. bsipocz

   added

   io.fits

   on Jun 17, 2018
3. bsipocz commented on Jun 17, 2018

   

   bsipocz

   on Jun 17, 2018

   Member

   More actions

   Ohh, actually, with a more careful look it seems that the tar file contains two fits files.

   kg:~/bsipocz/tegla2_mentes/munka/devel/astroquery/Maps/HST$ tar xvf eHst428013.tar 
   x HST/n48r020n0/n48r020n0_asn.fits.gz
   x HST/n48r020n0/n48r020n0_mos.fits.gz
   x HST/n48r020n0/n48r020n0_mos.jpg
   x HST/n48r020n0/n48r020n0_mos_thumb.jpg
   

   Reading them and accessing the header works perfectly fine. So this is an issue for astroquery that the esasky module should handle those tar files.

   For astropy.io.fits the task could be to provide a better error message that suggest that it wasn't really a fits file it tried to read in.
4. bsipocz

   added

   Bug

   on Jun 17, 2018
5. bsipocz

   changed the title [-]astroquery.esasky warning about astropy.io.fits.card[/-] [+]fits.open should gracefully exit when it's given a non FITS file[/+] on Jun 17, 2018
6. bsipocz

   changed the title [-]fits.open should gracefully exit when it's given a non FITS file[/-] [+]fits.open should raise useful exception when it's given a non FITS file[/+] on Jun 17, 2018
7. astrofrog commented on Jun 17, 2018

   

   astrofrog

   on Jun 17, 2018

   Last edited by astrofrog

   Member

   More actions

   I think all FITS files are required to start with SIMPLE - so maybe we could check for that? (note that we need to be careful about reading in compressed files though)

   👍React with 👍1Reacted by Simon Conseil
8. dhomeier

   closed this as completedin #10895on Nov 11, 2020