Unnecessary buffer allocations when writing base64 values with Utf8JsonWriter. #97628
Description
Description
Utf8JsonWriter.WriteBase64StringValue(ReadOnlySpan<byte>) implementation calls the following method to base64-encode the input bytes and write the result to the output buffer:
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private void Base64EncodeAndWrite(ReadOnlySpan<byte> bytes, Span<byte> output, int encodingLength)
{
byte[]? outputText = null;
Span<byte> encodedBytes = encodingLength <= JsonConstants.StackallocByteThreshold ?
stackalloc byte[JsonConstants.StackallocByteThreshold] :
(outputText = ArrayPool<byte>.Shared.Rent(encodingLength));
OperationStatus status = Base64.EncodeToUtf8(bytes, encodedBytes, out int consumed, out int written);
Debug.Assert(status == OperationStatus.Done);
Debug.Assert(consumed == bytes.Length);
encodedBytes = encodedBytes.Slice(0, written);
Span<byte> destination = output.Slice(BytesPending);
Debug.Assert(destination.Length >= written);
encodedBytes.Slice(0, written).CopyTo(destination);
BytesPending += written;
if (outputText != null)
{
ArrayPool<byte>.Shared.Return(outputText);
}
}The Base64EncodeAndWrite method allocates/rents the encodedBytes buffer, uses it to perform the base64 encoding then copies the contents to the output buffer. When you have a lot of concurrent writes (from different requests) and values larger than 256 bytes, the allocations can start to add up.
I think we can skip the temporary buffer and perform the encoding directly into the output buffer like in the snippet below:
private void Base64EncodeAndWrite(ReadOnlySpan<byte> bytes, Span<byte> output, int encodingLength)
{
Span<byte> destination = output.Slice(BytesPending);
OperationStatus status = Base64.EncodeToUtf8(
bytes,
destination,
out int consumed,
out int written
);
// removed Debug.Assert statements for brevity
BytesPending += written;
}If you think the proposed change is okay, I can go ahead and create a PR.
Configuration
.NET 8
Regression?
Data
Here's sample call stack where Base64EncodeAndWrite contributes to byte[] allocations in the large-object-heap.
