You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Dec 19, 2017. It is now read-only.
Repository navigation
This repository was archived by the owner on Dec 19, 2017. It is now read-only.
Hi,
I want to use the "ñ" character in a fit header. The verify in astropy is giving me an error.
WARNING: VerifyWarning: Card 'OBSERVER' is not FITS standard (invalid value string: 'Maria Rosa Mu�oz '). Fixed 'OBSERVER' card to meet the FITS standard. [astropy.io.fits.verify]
ERROR: UnicodeDecodeError: 'ascii' codec can't decode byte 0xf1 in position 1946: ordinal not in range(128) [astropy.io.fits.header]
Traceback (most recent call last):
File "./fixfitheader.py", line 30, in <module>
hdulist.writeto(sys.argv[2]+'/'+filename, output_verify='fix')
File "/usr/local/lib/python2.6/dist-packages/astropy/io/fits/hdu/hdulist.py", line 675, in writeto
hdu._writeto(hdulist.__file)
File "/usr/local/lib/python2.6/dist-packages/astropy/io/fits/hdu/base.py", line 497, in _writeto
self._writeheader(fileobj)
File "/usr/local/lib/python2.6/dist-packages/astropy/io/fits/hdu/base.py", line 428, in _writeheader
self._header.tofile(fileobj)
File "/usr/local/lib/python2.6/dist-packages/astropy/io/fits/header.py", line 625, in tofile
fileobj.write(blocks.encode('ascii'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf1 in position 1946: ordinal not in range(128)
Is there a solution for this issue?
Obviously PyFITS is correct to raise an exception here, but the specific exception given is not terribly useful. First of all, the VerificationWarning states: "Fixed 'OBSERVER' card to meet the FITS standard." which it clearly did not, because there's still a crash later one when trying to write out the header.
Need to decide what the correct behavior should be, but it should either raise an UnfixableError exception immediately upon validation, or it should have a default fix of removing (or possibly replacing) the offending character. There are a number of possible approaches for replacement--there do exist transliteration tables. But it's probably not worth the effort in an initial fix.
I was wondering how the user even got and ñ into the header in the first place, because normally the Card class prevents assignment of values that are not ASCII-encodable. But it turns out that on Python 2 if you pass in a str containing non-ASCII characters of questionable encoding it will allow that. So that is a secondary bug that should be fixed first.
Once that is fixed we still need to hack in a way to force non-ASCII characters into a header so that we can ensure that that case is handled better in case it somehow happens by accident as it did here.
I've fixed the issue that motivated this, that Python 2 was allowing non-ASCII values to slip by.
The bigger issue is still how to handle existing headers that contain non-ASCII characters. Fortunately this is generally only going to be an issue for files coming from the wild, and even there the issue has come up thankfully infrequently. Even when it does, this will generally result in a unicode decode error when trying to parse the file.
This does need better handling at the validation level, but that will need to be part of the broader validation overhaul that needs to be performed so I am going to put that off for the time being.
Activity
embray commentedon Feb 1, 2014
I was wondering how the user even got and ñ into the header in the first place, because normally the
Cardclass prevents assignment of values that are not ASCII-encodable. But it turns out that on Python 2 if you pass in astrcontaining non-ASCII characters of questionable encoding it will allow that. So that is a secondary bug that should be fixed first.Once that is fixed we still need to hack in a way to force non-ASCII characters into a header so that we can ensure that that case is handled better in case it somehow happens by accident as it did here.
Squashed commit of the following:
Squashed commit of the following:
embray commentedon Feb 1, 2014
I've fixed the issue that motivated this, that Python 2 was allowing non-ASCII values to slip by.
The bigger issue is still how to handle existing headers that contain non-ASCII characters. Fortunately this is generally only going to be an issue for files coming from the wild, and even there the issue has come up thankfully infrequently. Even when it does, this will generally result in a unicode decode error when trying to parse the file.
This does need better handling at the validation level, but that will need to be part of the broader validation overhaul that needs to be performed so I am going to put that off for the time being.
Squashed commit of the following: