Add Configuration for Compression in AzureStorageMessageDataRepository for Claim Check #5744
Conversation
…laim check Added a compression option to the AzureStorageMessageDataRepository class. Updated relevant methods to support compression. Modified configuration extensions to handle the new compression feature.
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
phatboyg
left a comment
phatboyg
left a comment
There was a problem hiding this comment.
Also, can you clone this unit test to verify the compression works?
|
|
||
| return await blob.OpenReadAsync(new BlobOpenReadOptions(false), cancellationToken).ConfigureAwait(false); | ||
| var stream = await blob.OpenReadAsync(new BlobOpenReadOptions(false), cancellationToken).ConfigureAwait(false); | ||
| return blobName.EndsWith(".gz", StringComparison.OrdinalIgnoreCase) || _compress ? new GZipStream(stream, CompressionMode.Decompress, true) : stream; |
There was a problem hiding this comment.
It seems like you're going to have some ownership/disposable issues with these streams to where they aren't closed and/or disposed properly.
There was a problem hiding this comment.
You are right, now when compressing I am setting the leaveOpen to false to dispose the stream after disposing the GZipStream object
| if (_compress) | ||
| { | ||
| blobName += ".gz"; | ||
| blob = _container.GetBlobClient(blobName); |
There was a problem hiding this comment.
Determine the blob name before calling GetBlobClient, and only call it once (the call for the original filename is above).
There was a problem hiding this comment.
This is done, take a look to last commit
|
Also, the documentation for claim check (message data) is way out of date. You mentioned you updated the documentation, but I didn't see any updates in that area. |
Adding unit tests to ensure compression is working. Fix ownership/dispose of the stream once it is compressed Avoid to create the BlobClient twice
I forgot to push my last commit, my bad, now it is updated |
celia-moreno
deleted the
cmo/Enable_compression_azure_storage_message_data_repository
branch
3 participants
This update enables compression when using AzureStorageMessageDataRepository for claim check.
Files will be compressed before being written to blob storage and saved as
.gzfiles.On the consumer side, there is no need to manually set the flag, as the system will automatically detect if the file in blob storage is a .gz file and decompress it accordingly.
Changes Summary:
Added a compression option to the AzureStorageMessageDataRepository class.
Updated relevant methods to support compression.
Modified configuration extensions to handle the new compression feature.
Additionally, the documentation has been updated to reflect the new flag when initializing the AzureStorageMessageDataRepository:
