Skip to content

.Net:[OpenAPI] Prevent buffering of streamed responses - #8557

Merged
SergeyMenshykh merged 3 commits into
microsoft:mainfrom
SergeyMenshykh:open-api-response-headers-read
Sep 9, 2024
Merged

SergeyMenshykh merged 3 commits into
microsoft:mainfrom
SergeyMenshykh:open-api-response-headers-read

Conversation

@SergeyMenshykh

@SergeyMenshykh SergeyMenshykh commented Sep 5, 2024 •

Copy link
Copy Markdown
Contributor

Motivation and Context

Today, the RestApiOperationRunner reads the entire response body into a memory buffer before the HttpClient.SendAsync operation returns control to the calling code. This approach is not efficient for scenarios where the calling code can start working with the stream content as soon as it becomes available, without waiting until it's fully read and buffered.

Description

This PR specifies the HttpCompletionOption.ResponseHeadersRead option as an argument of the HttpClient.SendAsync method to indicate that the method should return execution control to the calling code as soon as headers are available but before the content is read. If the calling code specifies a custom HTTP content reader to read the response content as a stream, then the calling code will start receiving the streamed content as soon as its first bytes are available. In all other cases, when the response content is consumed as a string using the ReadAsStringAsync method or as a byte array using the ReadAsByteArrayAsync method, the response content will be buffered in memory and become available when fully read from the wire.

Addresses: #8070 (comment)

Contribution Checklist

…on to prevent unnecessary HTTP response body buffering into memory.
@markwallace-microsoft Mark Wallace (markwallace-microsoft) added the .NET Issue or Pull requests regarding .NET code label Sep 5, 2024
@SergeyMenshykh SergeyMenshykh added the openapi Issues related to the OpenAPI function importer label Sep 5, 2024
@SergeyMenshykh
SergeyMenshykh added this pull request to the merge queue Sep 9, 2024
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 9, 2024
@SergeyMenshykh
SergeyMenshykh added this pull request to the merge queue Sep 9, 2024
github-merge-queue Bot pushed a commit that referenced this pull request Sep 9, 2024
### Motivation and Context
Today, the `RestApiOperationRunner` reads the entire response body into
a memory buffer before the `HttpClient.SendAsync` operation returns
control to the calling code. This approach is not efficient for
scenarios where the calling code can start working with the stream
content as soon as it becomes available, without waiting until it's
fully read and buffered.

### Description
This PR specifies the `HttpCompletionOption.ResponseHeadersRead` option
as an argument of the `HttpClient.SendAsync` method to indicate that the
method should return execution control to the calling code as soon as
headers are available but before the content is read. If the calling
code specifies a custom HTTP content reader to read the response content
as a stream, then the calling code will start receiving the streamed
content as soon as its first bytes are available. In all other cases,
when the response content is consumed as a string using the
`ReadAsStringAsync` method or as a byte array using the
`ReadAsByteArrayAsync` method, the response content will be buffered in
memory and become available when fully read from the wire.

Addresses:
#8070 (comment)

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 9, 2024
@SergeyMenshykh
SergeyMenshykh added this pull request to the merge queue Sep 9, 2024
Merged via the queue into microsoft:main with commit 24294bf Sep 9, 2024
@SergeyMenshykh
SergeyMenshykh deleted the open-api-response-headers-read branch September 9, 2024 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Issue or Pull requests regarding .NET code openapi Issues related to the OpenAPI function importer

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

5 participants