io.TextIOWrapper always closes wrapped files #65562

Activity

1. mitsuhiko commented on Apr 27, 2014

   

   mitsuhiko

   on Apr 27, 2014

   MemberAuthor

   More actions

   I'm trying to write some code that fixes a misconfigured sys.stdin on a case by case bases but unfortunately I cannot use TextIOWrapper for this because it always closes the underlying file:

   Python

   >>> import io
   >>> sys.stdin.encoding
   'ANSI_X3.4-1968'
   >>> stdin = sys.stdin
   >>> correct_stdin = io.TextIOWrapper(stdin.buffer, 'utf-8')
   >>> correct_stdin.readline()
   foobar
   'foobar\n'
   >>> del correct_stdin
   >>> stdin.readline()
   Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   ValueError: I/O operation on closed file.

   Ideally there would be a way to disable this behavior.
2. eryksun commented on Apr 27, 2014

   

   eryksun

   on Apr 27, 2014

   Contributor

   More actions

   It works if you detach the buffer beforehand:

       >>> import io, sys
       >>> stdin = sys.stdin
       >>> stdin.flush()
       >>> correct_stdin = io.TextIOWrapper(stdin.buffer, 'utf-8')
       >>> correct_stdin.readline()
       foobar
       'foobar\n'
   
       >>> correct_stdin.detach()
       <_io.BufferedReader name='<stdin>'>
   
       >>> del correct_stdin
       >>> stdin.readline()
       foobar
       'foobar\n'
3. bitdancer commented on May 14, 2014

   

   bitdancer

   on May 14, 2014

   Member

   More actions

   I think detach is the correct way to to this, so I'm closing the issue.
4. bitdancer

   closed this as completedon May 14, 2014
5. bitdancer

   added

   invalid

   on May 14, 2014
6. mitsuhiko commented on May 14, 2014

   

   mitsuhiko

   on May 14, 2014

   MemberAuthor

   More actions

   Detach "destroys" the stream, so it's not a solution. I can't just randomly destroy global state just because it's convenient.

   This is what I am doing now which seems borderline insane: https://github.com/mitsuhiko/click/blob/master/click/_compat.py#L31
7. mitsuhiko commented on May 14, 2014

   

   mitsuhiko

   on May 14, 2014

   MemberAuthor

   More actions

   Ah. Misread. This is about detaching the underlying stream from TextIOWrapper. I assume this could be done in the __del__ so that would work. I'm checking this now.
8. mitsuhiko commented on May 14, 2014

   

   mitsuhiko

   on May 14, 2014

   MemberAuthor

   More actions

   I can confirm that calling detach() in __del__ within an except block solves the issue.
9. mitar commented on Oct 20, 2018

   

   mitarmannequin

   on Oct 20, 2018

   Mannequin

   More actions

   I have a similar problem that text wrapper is closing the handle, and if I want to make a workaround, it also fails:

   buffer = io.Bytes()
   with io.TextIOWrapper(buffer, encoding='utf8') as text_buffer:
       write_content_to(text_buffer)
       text_buffer.flush()
       text_buffer.detach()

   Now this fails when context manager is trying to close the text_buffer with an error that it is already detached. If I do not detach it, then it closes buffer as well.
10. mhvk commented on Jun 3, 2021

    

    mhvkmannequin

    on Jun 3, 2021

    Mannequin

    More actions

    In astropy we are now working around the auto-closing of the underlying stream in TextIOWrapper by subclassing and overriding __del__ to detach [1]. It would seem more elegant if TestIOWrapper (and really, BufferedReader) could gain an closefd argument, just like open has, which is True by default.

    p.s. Do let me know if it is better to open a new issue.

    [1] astropy/astropy#11809
11. mhvk

    added

    type-featureA feature request or enhancementA feature request or enhancement

    on Jun 3, 2021
12. ezio-melotti

    transferred this issue fromon Apr 10, 2022