Skip to content

extmod/modssl_mbedtls: Implement SSLSession support. #12780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
DvdGiessen wants to merge 5 commits into
base: master
Choose a base branch
from
Open

extmod/modssl_mbedtls: Implement SSLSession support. #12780

DvdGiessen wants to merge 5 commits into from
+316 −7

Conversation

@DvdGiessen
Copy link
Contributor

@DvdGiessen DvdGiessen commented Oct 23, 2023 •
edited
Loading

Summary

This implements support for the SSLSession class, introduced in CPython in 3.6 (see #2415). It allows saving session data from an active TLS client-side connection and then creating a new connection re-using this session data. Benefits include a faster handshake and reduced data usage for short connections.

Implementation details

This PR adds the SSLSession class, the session= parameter for the SSLContext.wrap_socket() method, and the session attribute for an SSLSocket object.

Additionally, I've added a non-standard part: The SSLSession.serialize() function that converts the session to a bytes object (also available via the buffer protocol, so perhaps exposing this function is redundant); so that it can be stored by the user, and a constructor for the SSLSession object that accepts a bytes-like object to reconstruct the session object (CPython doesn't allow direct construction). This allows storing the session somewhere and use it after a deep sleep or reboot.

The second commit adds server-side support for TLS tickets in the Unix port, so that we can meaningfully test the session resumption in tests. The third commit adds a test which tests session resumption using the SSLSession object, checking that the resumption worked by checking that a resuming consumes less data.

micropython/micropython-lib#829 is a companion MR that implements support in the ssl module wrapper. It is required for the tests to pass.

Usage example

A small example test, using a wrapper class around the TCP socket so we can count how many bytes of data we're sending/receiving:

from io import IOBase
import socket
import ssl
import time

import network
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect('my-ssid', 'my-password')
while wlan.ifconfig()[0] == '0.0.0.0':
    time.sleep(0.1)

class AccountingStream(IOBase):
    def __init__(self, stream):
        self.stream = stream
        self.bytes_read = 0
        self.bytes_written = 0
        for attr in dir(stream):
            if not hasattr(self, attr):
                value = getattr(stream, attr)
                if callable(value):
                    setattr(self, attr, value)
    def read(self, size=None):
        result = self.stream.read() if size is None else self.stream.read(size)
        self.bytes_read += len(result)
        return result
    def readinto(self, buf, nbytes=None):
        result = self.stream.readinto(buf) if nbytes is None else self.stream.readinto(buf, nbytes)
        self.bytes_read += result
        return result
    def write(self, buf):
        self.bytes_written += len(buf)
        return self.stream.write(buf)

def connect_and_count(host, port, session=None):
    addr = socket.getaddrinfo(host, port)[0][-1]
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.settimeout(10)
    sock.connect(addr)
    sock_accounting = AccountingStream(sock)
    sock_tls = ssl.wrap_socket(sock_accounting, cert_reqs=ssl.CERT_NONE, server_hostname=host, session=session)
    sock_tls.write('HEAD / HTTP/1.1\r\nHost: {}\r\n\r\n'.format(host).encode())
    print('Response:', sock_tls.readline())
    print('Bytes read and written:', sock_accounting.bytes_read, sock_accounting.bytes_written)
    session = sock_tls.session
    sock.close()
    return session

host, port = 'tls-v1-2.badssl.com', 1012
socket.getaddrinfo(host, port)

print('Clean start:')
t = time.ticks_ms()
session = connect_and_count(host, port, None)
print('Time (ms):', time.ticks_diff(time.ticks_ms(), t))

print('\nReusing SSLSession:')
t = time.ticks_ms()
session = connect_and_count(host, port, session)
print('Time (ms):', time.ticks_diff(time.ticks_ms(), t))

print('\nUsing serialized and parsed SSLSession:')
t = time.ticks_ms()
session = connect_and_count(host, port, ssl.SSLSession(bytes(session)))
print('Time (ms):', time.ticks_diff(time.ticks_ms(), t))
Clean start:
Response: b'HTTP/1.1 200 OK\r\n'
Bytes read and written: 4947 453
Time (ms): 1829

Reusing SSLSession:
Response: b'HTTP/1.1 200 OK\r\n'
Bytes read and written: 438 602
Time (ms): 919

Using serialized and parsed SSLSession:
Response: b'HTTP/1.1 200 OK\r\n'
Bytes read and written: 438 602
Time (ms): 926

Testing

I've deployed this in production and been running it for a number of years on a large number of devices.

@DvdGiessen DvdGiessen marked this pull request as draft October 23, 2023 16:55
@github-actions
Copy link

github-actions bot commented Oct 23, 2023 •
edited
Loading

Code size report:

Reference:  all: Bump version to 1.27.0. [78ff170]
Comparison: extmod/modtls_mbedtls: Test SSLSession reuse. [merge of 88c4c9c]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64: +6240 +0.725% standard[incl +192(data)]
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2: +1304 +0.141% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@codecov
Copy link

codecov bot commented Oct 23, 2023 •
edited
Loading

Codecov Report

❌ Patch coverage is 90.90909% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.36%. Comparing base (78ff170) to head (88c4c9c).

Files with missing lines Patch % Lines
extmod/modtls_mbedtls.c 90.90% 5 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master   #12780      +/-   ##
==========================================
- Coverage   98.38%   98.36%   -0.02%     
==========================================
  Files         171      171              
  Lines       22300    22353      +53     
==========================================
+ Hits        21939    21987      +48     
- Misses        361      366       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dpgeorge dpgeorge added the extmod Relates to extmod/ directory in source label Nov 3, 2023
@Gadgetoid Gadgetoid mentioned this pull request Feb 29, 2024
Merged
@DvdGiessen DvdGiessen force-pushed the mbedtls_sslsession branch from 53bb552 to e529117 Compare March 4, 2024 17:07
@projectgus
Copy link
Contributor

projectgus commented Mar 7, 2024

This is an automated heads-up that we've just merged a Pull Request
that removes the STATIC macro from MicroPython's C API.

See #13763

A search suggests this PR might apply the STATIC macro to some C code. If it
does, then next time you rebase the PR (or merge from master) then you should
please replace all the STATIC keywords with static.

Although this is an automated message, feel free to @-reply to me directly if
you have any questions about this.

@DvdGiessen DvdGiessen force-pushed the mbedtls_sslsession branch 2 times, most recently from f014564 to 6c50ae1 Compare March 19, 2024 14:38
@DvdGiessen DvdGiessen mentioned this pull request Mar 19, 2024
Open
@DvdGiessen DvdGiessen marked this pull request as ready for review March 19, 2024 15:11
@DvdGiessen
Copy link
Contributor Author

DvdGiessen commented Mar 19, 2024 •
edited
Loading

Updated on latest master branch, added server-side support for TLS tickets to the Unix port, and added a test that checks (a) that SSLSession works and (b) that session resumption actually results in decreased data usage.

I've been using various versions of this patch for almost a year now to resume HTTPS connections without any trouble (though that might just be because I didn't try with many different configurations).

Marked as ready for review.

EDIT: And re-pushed because I forgot to add the documentation commit.

@DvdGiessen DvdGiessen force-pushed the mbedtls_sslsession branch 3 times, most recently from caeb380 to feae3a7 Compare March 20, 2024 12:57
@DvdGiessen DvdGiessen force-pushed the mbedtls_sslsession branch from feae3a7 to a7c1dc6 Compare May 27, 2024 12:36
@ccrighton
Copy link

ccrighton commented Jun 8, 2024 •
edited
Loading

Hi Daniël,

Do you plan to update the asyncio implementation to use this functionality? If that was done it would be a minor effort to add SSL session support to many micropython web libraries.

Currently, we session set up times in the order of 5 seconds on the PICO W, the lack of SSL session reuse is a showstopper for web apps using asyncio's new SSL support.

Cheers,

Charlie

@optimal-system
Copy link

optimal-system commented Jun 8, 2024

Hi Daniël,
Micro-controllers like the Rasperry Pico have a very interesting potential for IoT. To be controlled by a web interface they need a http library like Phew. To run the main program (to input data) and the web server (to output data) they need asyncio like the Phew implementation by ccrighton. To output data in a safe mode they also need httpS support. But when you put all that together it becomes very very sluggish ! (to toggle the built in LED it normally takes 25ms but with https and asyncio it takes more than 5000ms). All that to say there is very good reason to update the asyncio implementation...

@DvdGiessen
Copy link
Contributor Author

DvdGiessen commented Aug 4, 2024

I didn't look into how to use this with asyncio before. It appears that in CPython there is no interface to use SSLSessions with asyncio.

For the core support we'd need to for example add a session= parameter to asyncio.open_connection (here), similar to the server_hostname parameter. That would be a non-standard addition, so I'd prefer someone that's a bit more familiar with the asyncio ecosystem comments on whether this or some other solution would be preferable here.

For that reason I might prefer to split this into a separate follow-up PR, since it doesn't impact the changes proposed here and thus does not need to block considering / reviewing / merging this PR.

@ccrighton
Copy link

ccrighton commented Aug 4, 2024

@DvdGiessen Hi Daniël,

I agree that the change is best handled as a separate PR as it requires changing the asyncio implementation to add session to the wrap_socket call.

Cheers,

Charlie

@DvdGiessen DvdGiessen force-pushed the mbedtls_sslsession branch from 16223e9 to 4f8529d Compare May 14, 2025 14:34
@DvdGiessen DvdGiessen force-pushed the mbedtls_sslsession branch from 4f8529d to 3eca373 Compare May 20, 2025 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

extmod Relates to extmod/ directory in source

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@DvdGiessen @projectgus @ccrighton @optimal-system @dpgeorge