Description

I came across this while comparing behaviour of dtls-echo against #11943.

According to the Section 7.4.1.2 of TLS 1.2 RFC, the server should return a handshake failure if no matching ciphersuite is found (relevant part marked bold):

The cipher suite list, passed from the client to the server in ClientHello message contains the combinations of algorithms supported by the client in order of the client' preference (favorite choice first). Each cipher suite defines a exchange algorithm, a bulk encryption algorithm (including secret length), a MAC algorithm, and a PRF. The server will select a suite or, if no acceptable choices are presented, return a handshake failure alert and close the connection. If the list contains suites the server does not recognize, support, or wish to use, server MUST ignore those cipher suites, and process the ones as usual.

The dtls-echo server returns a close_notify (code 0) alert instead of a handshake_failure (code 40) alert.

Steps to reproduce the issue

Server setup:

1. In the Makefile, enable PSK and disable ECC for the server:

CFLAGS += -DDTLS_PSK
#CFLAGS += -DDTLS_ECC

2. Compile and start a native instance running dtls-echo example with PORT=tap0 make all term
3. Start the DTLS server on the instance with dtlss start
4. Get IP of the DTLS server with ifconfig

Client setup:

For the client, uncomment the CFLAGS += -DTINYDTLS_DEBUG line in dtls-echo Makefile to enable debug output.

1. In the Makefile, enable ECC and disable PSK for the client:

#CFLAGS += -DDTLS_PSK
CFLAGS += -DDTLS_ECC

2. Compile and start a native instance running dtls-echo example with PORT=tap1 make all term
3. Send test string to the server using the IP from step 4. of the server setup: dtlsc <server ip> TEST

Expected results

The debug output will print lots of things, but we are interested only in the last line:

[...]
Oct 01 16:24:16 DEBG receive unencrypted: (2 bytes):
00000000 02 28

The last byte (0x28) represents the value for handshake_failure (40) as described by the TLS RFC

Actual results

[...]
Oct 01 16:30:15 DEBG send unencrypted: (2 bytes):
00000000 02 00

The last byte (0x00) represents the value for close_notify (0).