Summary

Double-free occurs in OpenSSL QUIC implementation when handling connection reset with multiple concurrent connections.

Environment

• OpenSSL Version: 3.5.2
• Platform: Linux x86_64
• Compiler: GCC 13

Description

When running multiple QUIC connections and one connection gets reset by peer, AddressSanitizer detects a double-free error in OpenSSL's error handling code.

==3704879==ERROR: AddressSanitizer: attempting double-free on 0x5020008d6cd0 in thread T148 (cas_qm_signalin):
    #0 0x7ffff78fc4d8 in free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52
    #1 0x555555807a63 in ERR_pop_to_mark (/home/panys/Documents/dev/c/cas/build/cas_main+0x2b3a63) (BuildId: e12327b6521bb2941a9e15f56a36b6badb8b00f2)
    #2 0x5555557c018c in demux_recv (/home/panys/Documents/dev/c/cas/build/cas_main+0x26c18c) (BuildId: e12327b6521bb2941a9e15f56a36b6badb8b00f2)
    #3 0x5555557c0596 in ossl_quic_demux_pump (/home/panys/Documents/dev/c/cas/build/cas_main+0x26c596) (BuildId: e12327b6521bb2941a9e15f56a36b6badb8b00f2)
    #4 0x55555576f4d0 in ossl_quic_port_subtick (/home/panys/Documents/dev/c/cas/build/cas_main+0x21b4d0) (BuildId: e12327b6521bb2941a9e15f56a36b6badb8b00f2)
    #5 0x5555557c0a74 in qeng_tick (/home/panys/Documents/dev/c/cas/build/cas_main+0x26ca74) (BuildId: e12327b6521bb2941a9e15f56a36b6badb8b00f2)
    #6 0x55555576f73f in ossl_quic_reactor_tick (/home/panys/Documents/dev/c/cas/build/cas_main+0x21b73f) (BuildId: e12327b6521bb2941a9e15f56a36b6badb8b00f2)
    #7 0x555555768b98 in ossl_quic_write_flags (/home/panys/Documents/dev/c/cas/build/cas_main+0x214b98) (BuildId: e12327b6521bb2941a9e15f56a36b6badb8b00f2)
    #8 0x55555574ca1c in SSL_write_ex2 (/home/panys/Documents/dev/c/cas/build/cas_main+0x1f8a1c) (BuildId: e12327b6521bb2941a9e15f56a36b6badb8b00f2)
    #9 0x5555556d263a in quicWriteMessage /home/panys/Documents/dev/c/cas/src/library/utils/quic/quic_lifecycle.c:22
    #10 0x5555556d49c7 in quicMainThread /home/panys/Documents/dev/c/cas/src/library/utils/quic/quic_lifecycle.c:112
    #11 0x7ffff785ea41 in asan_thread_start ../../../../src/libsanitizer/asan/asan_interceptors.cpp:234
    #12 0x7ffff6c9caa3 in start_thread nptl/pthread_create.c:447
    #13 0x7ffff6d29c3b in clone3 ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

freed by thread T150 (cas_qm_0_0) here:
    #0 0x7ffff78fc4d8 in free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52
    #1 0x555555804bf1 in ERR_clear_error (/home/panys/Documents/dev/c/cas/build/cas_main+0x2b0bf1) (BuildId: e12327b6521bb2941a9e15f56a36b6badb8b00f2)

Analysis

The same memory is freed twice by two separate thread-safe QUIC I/O clients:

• cas_qm_0_0: A data exchange QUIC connection thread
• cas_qm_signalin: A signaling QUIC connection thread

Both are simple QUIC clients using default streams, implemented strictly following the OpenSSL QUIC non-blocking demo.

Architecture:

• Each QUIC client runs in its own thread handling read/write operations through a signaling mechanism
• Read signals come from a separate thread that performs select() on QUIC file descriptors (this thread never accesses OpenSSL resources directly)
• Write signals are triggered as needed during read/write cycles
• All OpenSSL operations are contained within their respective thread boundaries

Sequence of events:

1. cas_qm_signalin is a signaling connection that instructs the program to establish new QUIC connections (cas_qm_0_0) with the server for data exchange
2. When the server no longer needs data, it terminates the cas_qm_0_0 connection via SSL_ERROR_SSL with SSL_get_stream_read_state() == SSL_STREAM_STATE_RESET_REMOTE
3. During this connection reset cleanup process, cas_qm_signalin continues normal signaling operations
4. The double-free occurs during the concurrent error handling of the reset connection while the signaling connection remains active

Reproduction Steps

1. Setup: Create two concurrent QUIC connections following OpenSSL's non-blocking demo pattern:

   • One signaling connection (persistent)
   • One data connection (ephemeral)

2. Operation:

   • Keep signaling connection active with periodic message exchange
   • Establish data connection through signaling negotiation
   • Begin data transfer on the data connection

3. Trigger:

   • Have server initiate connection reset on data connection via stream reset
   • Ensure signaling connection continues normal operations during reset

4. Observe:

   • Double-free occurs in OpenSSL's error handling during cleanup
   • Race condition between ERR_clear_error() and ERR_pop_to_mark()

This suggests a race condition in OpenSSL's error state management when handling concurrent QUIC connection resets while other connections remain active.