Skip to content

feat(GCS+gRPC): upload checksums with last chunk#7031

Merged
coryan merged 2 commits intogoogleapis:mainfrom
coryan:feat-gcs-grpc-checksums-for-streaming-uploads
Jul 20, 2021
Merged

feat(GCS+gRPC): upload checksums with last chunk#7031
coryan merged 2 commits intogoogleapis:mainfrom
coryan:feat-gcs-grpc-checksums-for-streaming-uploads

Conversation

@coryan
Copy link
Copy Markdown
Contributor

@coryan coryan commented Jul 20, 2021

gRPC supports sending the object checksums in the last chunk of a
resumable upload. This changes the internal::ResumableUploadSession
class to receive these checksums, and weaves them through all the
decorators and tests. I also changed internal::ObjectWriteStream to
compute and store the checksums, and only validate them later.

Fixes #4156 and fixes #4157


This change is Reviewable

gRPC supports sending the object checksums in the last chunk of a
resumable upload. This changes the `internal::ResumableUploadSession`
class to receive these checksums, and weaves them through all the
decorators and tests. I also changed `internal::ObjectWriteStream` to
compute and store the checksums, and only validate them later.
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jul 20, 2021
@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: 38ba1f430581e8214747408adc54b810ec3a032f

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@codecov
Copy link
Copy Markdown

codecov Bot commented Jul 20, 2021

Codecov Report

Merging #7031 (8297b5c) into main (a3df9c7) will increase coverage by 0.00%.
The diff coverage is 97.57%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #7031   +/-   ##
=======================================
  Coverage   94.47%   94.47%           
=======================================
  Files        1305     1305           
  Lines      113350   113407   +57     
=======================================
+ Hits       107082   107139   +57     
  Misses       6268     6268           
Impacted Files Coverage Δ
.../storage/internal/curl_resumable_upload_session.cc 97.05% <ø> (ø)
...d/storage/internal/curl_resumable_upload_session.h 100.00% <ø> (ø)
...torage/internal/logging_resumable_upload_session.h 100.00% <ø> (ø)
...le/cloud/storage/internal/object_write_streambuf.h 100.00% <ø> (ø)
.../cloud/storage/internal/resumable_upload_session.h 60.00% <0.00%> (ø)
.../storage/internal/retry_resumable_upload_session.h 100.00% <ø> (ø)
google/cloud/storage/testing/mock_client.h 94.44% <ø> (ø)
.../storage/tests/object_checksum_integration_test.cc 100.00% <ø> (ø)
...loud/storage/tests/object_hash_integration_test.cc 100.00% <ø> (ø)
...e/tests/object_resumable_write_integration_test.cc 83.33% <0.00%> (-0.35%) ⬇️
... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3df9c7...8297b5c. Read the comment docs.

@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: 8297b5c9ac9a13d8b012e70e7494f9c423d04da0

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@coryan coryan marked this pull request as ready for review July 20, 2021 18:39
@coryan coryan requested a review from a team July 20, 2021 18:39
Copy link
Copy Markdown
Contributor

@devjgm devjgm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coryan coryan merged commit d508834 into googleapis:main Jul 20, 2021
@coryan coryan deleted the feat-gcs-grpc-checksums-for-streaming-uploads branch July 20, 2021 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement MD5 checksum in GrpcResumableUploadSession Implement CRC32C checksum in GrpcResumableUploadSession

4 participants