Repository navigation
Unexpected behavior in get_readable_fileobj in Py3 #6460
Description
Not sure if this is related to #3013, but the behavior in Py3 below does not make sense to me. The docstring claims one can input a File-like object, but that basically doesn't work at all as far as I can tell.
In [20]: fh = open('test5.dat', 'rb')
In [21]: type(fh.read(20))
Out[21]: bytes
In [22]: with get_readable_fileobj(fh) as fh2:
...: print(type(fh2.read(20)))
...:
<class 'str'> # EXPECTED bytes
In [23]: fh = open('test5.dat', 'r')
In [24]: type(fh.read(20))
Out[24]: str
In [25]: with get_readable_fileobj(fh) as fh2:
...: print(type(fh2.read(20)))
...:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-25-5b2dfcd6aacc> in <module>()
1 with get_readable_fileobj(fh) as fh2:
----> 2 print(type(fh2.read(20)))
3
AttributeError: '_io.TextIOWrapper' object has no attribute 'readinto'
# WHY THE EXCEPTION?
In [27]: fh = open('test5.dat', 'r')
In [28]: with get_readable_fileobj(fh) as fh2:
...: print(type(fh2.read()))
...:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-28-06b67839537f> in <module>()
1 with get_readable_fileobj(fh) as fh2:
----> 2 print(type(fh2.read()))
3
TypeError: read() should return bytes
# WTF?
Activity
Work around for astropy#6460
Work around for astropy#6460
pllim commentedon Aug 23, 2017
Did you try passing in
encoding='binary'intoget_readable_fileobj()?Work around for astropy#6460
Work around for astropy#6460
Work around for astropy#6460
Work around for astropy#6460
Work around for astropy#6460
Work around for astropy#6460
pllim commentedon Feb 27, 2018
Two things here...
get_readable_fileobj()does returnbytesif you explicitly gives itencoding='binary'. DefaultNonemakes it guess, and on my machine (and as yours above), it guessed wrong (utf-8).When in the "text" mode (as opposed to binary), the logic is basically
fh2 = io.TextIOWrapper(io.BufferedReader(fh), encoding='utf-8'). You can recreate the sameAttributeErrorjust using this and thenfh2.read(20).Not sure what we can do about these. Any idea?
pllim commentedon Oct 30, 2019
The behavior has changed since this was first reported.
now gives this exception:
If you use
encoding='binary'specifically, it behaves as you expected: