This is a copy of the same bug opened in GnuTLS since, as it is a problem of interaction between both, we are not sure which one may be the culprit.

Description of problem:

TLS handshake fails when connecting to specific HTTPS servers after upgrading OpenSSL from 3.5.4 to 3.6.0.

It is definitely a problem caused by updating OpenSSL to 3.6.0 because if you roll back to 3.5.4, everything works again.

This was initially detected in WebKit applications, so there is a related bug in Bugzilla.

Versions

OpenSSL: 3.6.0
GnuTLS: 3.8.10

Distributor of OpenSSL and GnuTLS

Arch Linux

How reproducible:

1. Use Arch Linux (up to date as of October 2025).
2. Ensure OpenSSL 3.6.0 is installed.
3. Install and configure nginx with a simple self-signed TLS certificate.
4. Launch any GnuTLS client such Epiphany, wget or aria2.
5. Open https://localhost/.

Actual results:

Epiphany shows an SSL/TLS error 'Peer failed to perform TLS handshake: Error decoding the received TLS packet.'

In wget or aria2 similar error 'GnuTLS: Error decoding the received TLS packet'.

Expected results:

Everything should work fine, regardless of the version of OpenSSL installed.