Skip to content

Commit 2d07272

Browse files
committed
Handle non-ASCII characters in FITS header
Fixes astropy#6074 and others. The non-ASCII characters are replaced by the U+FFFD REPLACEMENT CHARACTER, which will then emit verification errors but also allow to fix the problematic cards.
1 parent 878184f commit 2d07272

2 files changed

Lines changed: 15 additions & 1 deletion

File tree

‎CHANGES.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2288,6 +2288,9 @@ Other Changes and Additions
22882288

22892289
- Fix use of quantize level parameter for ``CompImageHDU``. [#6029]
22902290

2291+
- Prevent crash when a header contains non-ASCII (e.g. UTF-8) characters, to
2292+
allow fixing the problematic cards. [#6084]
2293+
22912294
- ``astropy.io.misc``
22922295

22932296
- ``astropy.io.registry``

‎astropy/io/fits/py3compat.py‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
# Licensed under a 3-clause BSD style license - see PYFITS.rst
22

3+
import warnings
34
from ...extern import six
45
from ...utils.compat.numpycompat import NUMPY_LT_1_10
6+
from ...utils.exceptions import AstropyUserWarning
57

68
if not six.PY2:
79
# Stuff to do if Python 3
@@ -27,7 +29,16 @@ def encode_ascii(s):
2729

2830
def decode_ascii(s):
2931
if isinstance(s, bytes):
30-
return s.decode('ascii')
32+
try:
33+
return s.decode('ascii')
34+
except UnicodeDecodeError:
35+
warnings.warn('non-ASCII characters are present in the FITS '
36+
'file header and have been replaced by the '
37+
'U+FFFD REPLACEMENT CHARACTER',
38+
AstropyUserWarning)
39+
return s.decode('ascii', errors='replace')
40+
# FIXME: replace with question mark ?
41+
# return s.replace(u'\ufffd', '?')
3142
elif (isinstance(s, numpy.ndarray) and
3243
issubclass(s.dtype.type, numpy.bytes_)):
3344
# np.char.encode/decode annoyingly don't preserve the type of the

0 commit comments

Comments
 (0)