Hi,

Thanks to the awesome OpenSSL community. I have successfully enable the TLS1.3 + early data in my system! Awesome!

However, when I try to do some microbenchmark to this fancy new TLS feature, I find enabling early data turns out consume more time to get the server response :(

Later I try to do a simple test by using OpenSSL's example code and find this issue still exists! Following is my settings and results of this small test:

Test settings

1. Testing environment: VMWare + Ubuntu 16.04
2. Testing OpenSSL branch: master (so, TLS 1.3 draft 20)
3. Socket delay: 50ms (i.e., sudo tc qdisc add dev lo root netem delay 50ms)
4. code modification: I just copy the early data read feature from s_server.c to its www_body(). Please check the reference in the bottom of this post for knowing the added code
5. server testing script: openssl s_server -key key.pem -cert cert.pem -accept 44330 -state -www -early_data
6. client testing script: openssl s_client -state -connect 127.0.0.1:44330 -keylogfile sslkeylog.log -sess_out session.dat -sess_in session.dat -early_data http.txt
   (http.txt is just a dummy GET request)

Testing results

1. server can get the early data correctly and write the early return back
2. client can ensure that the early data has been accepted and get the dummy early return (i.e., dummy20early)
3. Here is the result of wireshark traces: http://imgur.com/xt5vB4j.png

My problem

1. As shown in the attached figure of wireshark traces, there is no 0RTT. The early return basically wait more than 300ms to be received (instead of 1RTT = 100ms).
2. There is a huge delay (150ms) between the early data and the following packets sent to the server (I guess that is the client finish and the early data end). Should they be sent back-to-back?
3. Server needs to wait the session ticket being sent before sending the early reply by SSL_write_early_data(). Is it necessary?
4. Why not the client hello and early data being squeezed in the same packet? It seems the early data is sent only after the ACK of the client hello received, so no 0RTT for sure.

The above issues seems to make the TLS1.3 + early data not that useful. I haven't been able to check how the statem works when the early data is sent, but this testing result seems wrong to me. Is it something have not been explored by the OpenSSL community? or it is my fault to use OpenSSL in a wrong way?

(I would be happy to help fixing this problem, but I might need some helps)

Reference:

code added before the for(;;) reading loop of www_body in s_server.c
This code is nothing special, just return a dummy html page "dummy20early" when the server gets the early data sent from the client

// *** ADDED BY YCTUNG ***
    BIO_printf(bio_s_out, "early_data flag = %d\n", early_data);
    if (early_data) {
        BIO_printf(bio_s_out, "Going to check early data\n");
        int write_header = 1, edret = SSL_READ_EARLY_DATA_ERROR;
        size_t readbytes;

        while (edret != SSL_READ_EARLY_DATA_FINISH) {
            for (;;) {
                edret = SSL_read_early_data(con, buf, bufsize, &readbytes);
                if (edret != SSL_READ_EARLY_DATA_ERROR)
                    break;

                switch (SSL_get_error(con, 0)) {
                case SSL_ERROR_WANT_WRITE:
                case SSL_ERROR_WANT_ASYNC:
                case SSL_ERROR_WANT_READ:
                    /* Just keep trying - busy waiting */
                    continue;
                default:
                    BIO_printf(bio_err, "Error reading early data\n");
                    ERR_print_errors(bio_err);
                    goto err;
                }
            }
            if (readbytes > 0) {
                if (write_header) {
                    BIO_printf(bio_s_out, "Early data received, readbytes = %ld\n", readbytes);
                    write_header = 0;
                }
                raw_write_stdout(buf, (unsigned int)readbytes);
                (void)BIO_flush(bio_s_out);
            }
        }
        if (SSL_is_init_finished(con))
            print_connection_info(con);

        BIO_printf(bio_s_out, "edret = %d\n", edret);
        if (write_header)
            BIO_printf(bio_s_out, "No early data received\n");
        else {
            BIO_printf(bio_s_out, "\nEnd of early data, readbytes = %ld\n\n", readbytes);char body[]="<HTML><BODY>dummy20early</BODY></HTML>\r\n\r\n";
            int bodySize = sizeof(body) - 1;

            char resp[500];
            int respSize = -1;
            respSize = sprintf(resp, "HTTP/1.0 200 ok\r\nContent-type: text/html\r\nContent-Length: %d\r\n\r\n%s", bodySize, body);
            BIO_printf(bio_s_out, "*** early resp size = %d, data =\n%s\n", respSize, resp);
            
            // quickly return it by early data write
            size_t byteWritten = -1;
            edret = SSL_write_early_data(con, resp, respSize, &byteWritten);
            BIO_flush(io);
     }
}