This code only just got added recently. Why are you taking it out again?

yeah, sorry that I did not notice earlier, but the sockets here are all blocking.
implementing non-blocking sockets would be much more effort.

for instance the SSL_write above...

I'm not sure I understand. SSL_ERROR_WANT_READ can occur with blocking sockets - and does so in the TLSv1.3 case (which is why this change was made).

Apparently I have not tried with TLSv1.3, sorry.
Can you please explain why the SSL not simply reads the data?

Hmm...

I tried this but it did not help:

diff --git a/apps/s_socket.c b/apps/s_socket.c
index 804ab5b..5b0e141 100644
--- a/apps/s_socket.c
+++ b/apps/s_socket.c
@@ -179,7 +179,8 @@ int do_server(int *accept_sock, const char *host, const char
     asock = BIO_socket(BIO_ADDRINFO_family(res), BIO_ADDRINFO_socktype(res),
                        BIO_ADDRINFO_protocol(res), 0);
     if (asock == INVALID_SOCKET
-        || !BIO_listen(asock, BIO_ADDRINFO_address(res), BIO_SOCK_REUSEADDR)) {
+        || !BIO_listen(asock, BIO_ADDRINFO_address(res),
+                       BIO_SOCK_REUSEADDR | BIO_SOCK_NODELAY)) {
         BIO_ADDRINFO_free(res);
         ERR_print_errors(bio_err);
         if (asock != INVALID_SOCKET)

still tls1.2 -www -reuse
28410 connections in 13.43s; 2115.41 connections/user sec, bytes read 104177885
28410 connections in 31 real seconds, 3666 bytes read per connection

and tls1.3 -www -reuse
8615 connections in 15.03s; 573.19 connections/user sec, bytes read 38285060
8615 connections in 31 real seconds, 4444 bytes read per connection

so my conclusion is that reused sessions in tls1.3
cause the same cpu cycles for the client than new tls1.3 sessions,
just the server has significantly less cpu cycles.
OTOH tls1.2 session reuse may cause more round trips,
but much less cpu load on both server and client, and if
the network latency is very low as in this case the numbers
are not as convincing as they should be.
Is this by design, or could the TLS session have an
estimation of the network delay and choose another
strategy, if the round trips are cheap?

./openssl s_server -www
vs.
./openssl s_time -new
=>
5134 connections in 8.30s; 618.55 connections/user sec, bytes read 0
5134 connections in 31 real seconds, 0 bytes read per connection
./openssl s_time -reuse
=> (as expected SSL_session_reused() = 0)
5049 connections in 8.36s; 603.95 connections/user sec, bytes read 0
5049 connections in 31 real seconds, 0 bytes read per connection

These numbers are as expected, i.e. we get less connections/sec with the attempted reuse than without. In both cases we are doing a full handshake (we're not actually reusing)...but we've gone through some of the hoops of attempting to reuse. At least that's my guess.

./openssl s_time -www /test -new
=>
4435 connections in 8.31s; 533.69 connections/user sec, bytes read 21953250
4435 connections in 31 real seconds, 4950 bytes read per connection

Also expected. Less connections/sec because now we are actually transferring data.

Note: the connections/user sec are almost identical, does that mean the client
has no less work to do for a reuse, but the server ?

I'm not sure what you mean. So far in the above numbers no reuse has actually happened (SSL_session_reused() == 0)

./openssl s_time -www /test -reuse
=> (as expected SSL_session_reused() = 1)
8734 connections in 14.90s; 586.17 connections/user sec, bytes read 38840098
8734 connections in 31 real seconds, 4447 bytes read per connection

For the first time connections/sec goes up because we are now really doing reuse, so this is as expected.

now I start the server with tls1_2
./openssl s_server -www -tls1_2

./openssl s_time -new
=>
4626 connections in 7.92s; 584.09 connections/user sec, bytes read 0
4626 connections in 31 real seconds, 0 bytes read per connection

./openssl s_time -reuse
=>
40253 connections in 16.34s; 2463.46 connections/user sec, bytes read 0
40253 connections in 31 real seconds, 0 bytes read per connection
Note: This is the only data point which is how I would expect it.

./openssl s_time -www /test -new
=>
4527 connections in 7.85s; 576.69 connections/user sec, bytes read 22413177
4527 connections in 31 real seconds, 4951 bytes read per connection

All of the above are largely as expected. Session reuse is faster than no session reuse. TLSv1.3 without session reuse is out performing TLSv1.2 without session reuse.

Slightly surprising is that TLSv1.2 www without session reuse is marginally better than the TLSv1.3 equivalent.

./openssl s_time -www /test -reuse
=> (yes SSL_session_reused() = 1 here)
725 connections in 0.87s; 833.33 connections/user sec, bytes read 2657125
725 connections in 31 real seconds, 3665 bytes read per connection

This one I cannot explain. TLSv1.2 www with session reuse is much worse than without???

Slightly surprising is that TLSv1.2 www without session reuse is marginally better than the TLSv1.3 equivalent.

Perhaps this could be to do with the cipher that is being selected in each case?

./openssl s_time -www /test -reuse
=> (yes SSL_session_reused() = 1 here)
725 connections in 0.87s; 833.33 connections/user sec, bytes read 2657125
725 connections in 31 real seconds, 3665 bytes read per connection

This one I cannot explain. TLSv1.2 www with session reuse is much worse than without???

Yes, that was before I added the NODELAY flags.
with NODELAY the same test results in much better numbers:

still tls1.2 -www -reuse
28410 connections in 13.43s; 2115.41 connections/user sec, bytes read 104177885
28410 connections in 31 real seconds, 3666 bytes read per connection

that was the only data point that changed due to NODELAY.

Why not make portion that makes the call to a block and declare no_linger there?

I mean this way there would be one #ifdef...

done.

Formally speaking it might be more appropriate to implement it as BIO_CTRL. I mean whole point with BIO is to arrange implementation-neutral interface, and this kind of very specific. On the other hand this is special case...

Yes, kids, don't do that at home ;-)

You don't really need else here. Starting with straight { works and I'd say is more readable in this case.

well, a matter of taste, but if you like it better without else it's ok for me.

Some even "hate the else after return" :-) But to me, in this case, it's really else in #if. I'm not actually a stranger to such construct (I mean I've used it myself), but it's always "unless actually necessary". And it's not actually necessary in this case...