Skip to content

QuicStream leaks its native QUIC_BUFFER array for every stream that sends data #135583

Closed

Description

@WilliamSmithEdward

Description

Every QuicStream that sends data leaks one 16-byte native allocation (its
QUIC_BUFFER array). The array is never freed when the stream is disposed. On a
long-running HTTP/3 server this is one leak per request, and native memory
grows without bound. We first saw it as roughly 80-110 MiB/hour of private-byte
growth in an HTTP/3 endurance run. The leak is the same with Kestrel and with
our own server (EmbedIO), and it does not occur for HTTP/1.1 or HTTP/2.

Cause

MsQuicBuffers is a struct, and QuicStream hands a copy of it to its handle:

private MsQuicBuffers _sendBuffers = new MsQuicBuffers();
...
_handle = new MsQuicContextSafeHandle(handle, context, SafeHandleType.Stream, connectionHandle)
{
    Disposable = _sendBuffers   // boxes a copy while _buffers is still null
};

The first WriteAsync calls _sendBuffers.Initialize(buffer) on the field
(#L412).
Reserve(1) then allocates the array with NativeMemory.AllocZeroed into the
field
(MsQuicBuffers.cs#L40).
SEND_COMPLETE calls Reset(), which frees the data buffers but keeps the
array for reuse. When the handle is released it disposes the boxed copy
(MsQuicSafeHandle.cs#L143).
That copy's _buffers is still null, so FreeNativeMemory frees nothing, and
the field's array is orphaned.

Nothing else frees the field, and main has the same code (QuicStream.cs
lines 112, 211, 245 and 450 at 8630d89).

Reproduction Steps

Windows (the heap walk is the only Windows-specific part; the reflection check
works anywhere). Console app, net10.0, no packages.

Program.cs
using System.Net;
using System.Net.Quic;
using System.Net.Security;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;

#pragma warning disable CA1416 // Windows only: the heap walk below.
var alpn = new SslApplicationProtocol("repro");
using var key = RSA.Create(2048);
var request = new CertificateRequest("CN=localhost", key, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);
using var ephemeral = request.CreateSelfSigned(DateTimeOffset.UtcNow.AddDays(-1), DateTimeOffset.UtcNow.AddDays(1));
using var certificate = X509CertificateLoader.LoadPkcs12(ephemeral.Export(X509ContentType.Pkcs12), null);

await using var listener = await QuicListener.ListenAsync(new QuicListenerOptions
{
    ListenEndPoint = new IPEndPoint(IPAddress.Loopback, 0),
    ApplicationProtocols = [alpn],
    ConnectionOptionsCallback = (_, _, _) => ValueTask.FromResult(new QuicServerConnectionOptions
    {
        DefaultStreamErrorCode = 1,
        DefaultCloseErrorCode = 1,
        ServerAuthenticationOptions = new SslServerAuthenticationOptions { ApplicationProtocols = [alpn], ServerCertificate = certificate },
    }),
});
var accept = listener.AcceptConnectionAsync();
await using var client = await QuicConnection.ConnectAsync(new QuicClientConnectionOptions
{
    RemoteEndPoint = listener.LocalEndPoint,
    DefaultStreamErrorCode = 1,
    DefaultCloseErrorCode = 1,
    ClientAuthenticationOptions = new SslClientAuthenticationOptions
    {
        ApplicationProtocols = [alpn],
        TargetHost = "localhost",
        RemoteCertificateValidationCallback = (_, _, _, _) => true,
    },
});
await using var server = await accept;

var sendBuffers = typeof(QuicStream).GetField("_sendBuffers", BindingFlags.Instance | BindingFlags.NonPublic)!;
var buffer = new byte[1];
foreach (var write in new[] { true, false })
{
    var before = Busy16();
    var unfreed = 0;
    for (var i = 0; i < 2000; i++)
    {
        await using var outbound = await client.OpenOutboundStreamAsync(QuicStreamType.Bidirectional);
        outbound.CompleteWrites();
        var inbound = await server.AcceptInboundStreamAsync();
        while (await inbound.ReadAsync(buffer) > 0) { }
        if (write) await inbound.WriteAsync(new byte[] { 42 }, completeWrites: true);
        else inbound.CompleteWrites();
        while (await outbound.ReadAsync(buffer) > 0) { }
        await inbound.DisposeAsync();

        // MsQuicBuffers.FreeNativeMemory resets _count together with _buffers.
        var copy = sendBuffers.GetValue(inbound)!;
        if ((int)copy.GetType().GetField("_count", BindingFlags.Instance | BindingFlags.NonPublic)!.GetValue(copy)! != 0) unfreed++;
    }

    Console.WriteLine($"{(write ? "1-byte write" : "empty write ")}: 2000 disposed streams, {unfreed} still own a QUIC_BUFFER array, live 16-byte heap blocks {Busy16() - before:+#;-#;0}");
}

static long Busy16()
{
    for (var i = 0; i < 3; i++) { GC.Collect(); GC.WaitForPendingFinalizers(); }
    var heap = GetProcessHeap();
    long count = 0;
    HeapLock(heap);
    try
    {
        var entry = default(HeapEntry);
        while (HeapWalk(heap, ref entry)) if ((entry.Flags & 4) != 0 && entry.Size == 16) count++;
    }
    finally { HeapUnlock(heap); }
    return count;
}

[DllImport("kernel32.dll")] static extern IntPtr GetProcessHeap();
[DllImport("kernel32.dll")] static extern bool HeapLock(IntPtr heap);
[DllImport("kernel32.dll")] static extern bool HeapUnlock(IntPtr heap);
[DllImport("kernel32.dll")] static extern bool HeapWalk(IntPtr heap, ref HeapEntry entry);

[StructLayout(LayoutKind.Sequential)]
struct HeapEntry { public IntPtr Data; public uint Size; public byte Overhead; public byte Region; public ushort Flags; public IntPtr U0, U1, U2; }

Expected behavior

Disposing a stream frees its QUIC_BUFFER array. Both lines should report 0
retained arrays and no growth in live 16-byte heap blocks.

Actual behavior

1-byte write: 2000 disposed streams, 2000 still own a QUIC_BUFFER array, live 16-byte heap blocks +2000
empty write : 2000 disposed streams, 0 still own a QUIC_BUFFER array, live 16-byte heap blocks 0

Streams that complete with an empty write never call Initialize, so they do
not leak. That isolates the leak to the send-buffer array.

In the HTTP/3 endurance runs, a heap walk of the default process heap at
quiesced points showed one extra live 16-byte block per completed request:
1,000-1,010 per 1,000 requests, with Kestrel and with EmbedIO alike. Committed
heap memory grew about twice as fast as live bytes, which is consistent with
per-block heap overhead. The same growth occurred with app-local MsQuic 2.5.11
and 2.6.2 (via System.Net.Quic.AppLocalMsQuic), as expected for a leak in
the managed wrapper.

Regression?

Yes, it appears to be. #100228 ("Delay MsQuicSendBuffers disposal to QuicStream
handle closing", March 2024) replaced a direct _sendBuffers.Dispose() in
QuicStream with the Disposable = _sendBuffers handoff. release/9.0 and
release/10.0 contain the handoff; release/8.0 still disposes the field
directly. I have measured only .NET 10.0.12.

Known Workarounds

None at the application level; every stream that sends data allocates the array.

Configuration

  • .NET 10.0.12 (SDK 10.0.401), Windows 11 Pro 10.0.26300, x64
  • Bundled msquic.dll 2.5.10 (9ff06b71fd4b4d5258361598ada5b24cbc1beb20)

Other information

A possible fix is to dispose the field itself when the stream handle is
released, not a boxed copy. Alternatives are a reference type for the send
buffers, or freeing _sendBuffers explicitly once the handle is closed and no
further SEND_COMPLETE can arrive.

Activity

  1. dotnet-policy-service commented on Oct 11, 2026

    @dotnet-policy-service
    Contributor

    Tagging subscribers to this area: @karelz, @dotnet/ncl
    See info in area-owners.md if you want to be subscribed.

  2. asad24-dev commented on Oct 11, 2026

    @asad24-dev
    Contributor

    I'd like to work on this. I've traced the reported leak to the boxed copy of MsQuicBuffers and have a candidate fix that converts it to a sealed class, preserving the existing safe-handle disposal lifecycle, along with a regression test.

  3. self-assigned this
    on Oct 12, 2026
  4. added a commit that references this issue on Oct 13, 2026
    cba3c1d
  5. locked and limited conversation to collaborators on Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions