Skip to content

Python 3: wsgi doesn't handle correctly partial write of socket send() when using writelines() #295

@vstinner

Description

@vstinner

The wsgi module uses "wfile = conn.makefile('wb', self.wbufsize)" to get a file-like API and then call wfile.writelines(lines) to write data. If a socket send() is a partial write, writelines() doesn't retry and simply try to write the next line. In this case, the HTTP client is stuck and wait forever. See the following message for a script to reproduce the bug:
#274 (comment)

raylu wrote that it's a bug in CPython, but I don't think so. On Python 3, SocketIO.write() is clear on how it handles partial write:

    """Write the given bytes or bytearray object *b* to the socket
    and return the number of bytes written.  This can be less than
    len(b) if not all data could be written.  If the socket is
    non-blocking and no bytes could be written None is returned.
    """

Moreover, socket.makefile() doc of Python 3 says "The socket must be in blocking mode":
https://docs.python.org/dev/library/socket.html#socket.socket.makefile

eventlet monkey-patching should do something to use its own GreenPipe object. Hack to workaround this bug:

diff --git a/eventlet/greenio/base.py b/eventlet/greenio/base.py
index 5f0108c..204e6f0 100644
--- a/eventlet/greenio/base.py
+++ b/eventlet/greenio/base.py
@@ -289,8 +289,10 @@ class GreenSocket(object):
         return newsock

     if six.PY3:
-        def makefile(self, *args, **kwargs):
-            return _original_socket.makefile(self, *args, **kwargs)
+        def makefile(self, mode, *args, **kwargs):
+            from eventlet.greenio.py3 import GreenFileIO
+            fd = os.dup(self.fileno())
+            return GreenFileIO(fd, mode, *args, **kwargs)
     else:
         def makefile(self, *args, **kwargs):
             dupped = self.dup()

I don't think that this patch is correct because I'm not sure that it creates a buffered reader/writer, nor that it's the best design to duplicate the file descriptor of the socket.

On Python 2, makefile() creates a socket._filesocket object. This object calls sendall() and not send() in its flush() method. It works because sendall() is an eventlet method which handles partial write and retries to ensure that all bytes are written.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions