Repository navigation
.Net:[OpenAPI] Prevent buffering of streamed responses - #8557
Merged
SergeyMenshykh merged 3 commits intoSep 9, 2024
Merged
SergeyMenshykh merged 3 commits into
SergeyMenshykh merged 3 commits into
Conversation
…on to prevent unnecessary HTTP response body buffering into memory.
SergeyMenshykh
commented
Sep 5, 2024
westey (westey-m)
approved these changes
Sep 6, 2024
Roger Barreto (rogerbarreto)
approved these changes
Sep 9, 2024
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Sep 9, 2024
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Sep 9, 2024
### Motivation and Context Today, the `RestApiOperationRunner` reads the entire response body into a memory buffer before the `HttpClient.SendAsync` operation returns control to the calling code. This approach is not efficient for scenarios where the calling code can start working with the stream content as soon as it becomes available, without waiting until it's fully read and buffered. ### Description This PR specifies the `HttpCompletionOption.ResponseHeadersRead` option as an argument of the `HttpClient.SendAsync` method to indicate that the method should return execution control to the calling code as soon as headers are available but before the content is read. If the calling code specifies a custom HTTP content reader to read the response content as a stream, then the calling code will start receiving the streamed content as soon as its first bytes are available. In all other cases, when the response content is consumed as a string using the `ReadAsStringAsync` method or as a byte array using the `ReadAsByteArrayAsync` method, the response content will be buffered in memory and become available when fully read from the wire. Addresses: #8070 (comment) ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Sep 9, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Today, the
RestApiOperationRunnerreads the entire response body into a memory buffer before theHttpClient.SendAsyncoperation returns control to the calling code. This approach is not efficient for scenarios where the calling code can start working with the stream content as soon as it becomes available, without waiting until it's fully read and buffered.Description
This PR specifies the
HttpCompletionOption.ResponseHeadersReadoption as an argument of theHttpClient.SendAsyncmethod to indicate that the method should return execution control to the calling code as soon as headers are available but before the content is read. If the calling code specifies a custom HTTP content reader to read the response content as a stream, then the calling code will start receiving the streamed content as soon as its first bytes are available. In all other cases, when the response content is consumed as a string using theReadAsStringAsyncmethod or as a byte array using theReadAsByteArrayAsyncmethod, the response content will be buffered in memory and become available when fully read from the wire.Addresses: #8070 (comment)
Contribution Checklist