Skip to content

feat(storage)!: move GCS+gRPC to v2/ protos#7254

Merged
coryan merged 4 commits intogoogleapis:mainfrom
coryan:feat-gcs-grpc-move-implementation-to-v2-pr1-no-emulator
Sep 2, 2021
Merged

feat(storage)!: move GCS+gRPC to v2/ protos#7254
coryan merged 4 commits intogoogleapis:mainfrom
coryan:feat-gcs-grpc-move-implementation-to-v2-pr1-no-emulator

Conversation

@coryan
Copy link
Copy Markdown
Contributor

@coryan coryan commented Sep 2, 2021

GCS+gRPC has published a new set of protos to target. These protos
rename and change the types for a number of fields, and they only
describe the RPCs that are actually implemented.

The major modifications in this change include:

  • The type and names of the crc32c and md5_hash fields changed.
  • Previously the md5 hash was sometimes represented as a hex-encoded
    binary blob and sometimes as just a binary blob. Now it is always
    a binary blob.
  • Several fields (like if_generation_match and crc32c) are now
    marked as optional in the protos, instead of using *Value
    message wrappers.
  • The proto version of ObjectAccessControl does not contain the
    bucket, object or generation fields.
  • Likewise, BucketAccessControl does not contain the bucket name.
  • We use https://github.com/googleapis/storage-testbench as it
    supports the v2/ protos.

Part of the work for #6982 Fixes #6643 Fixes #6641


This change is Reviewable

GCS+gRPC has published a new set of protos to target. These protos
rename and change the types for a number of fields, and they *only*
describe the RPCs that are actually implemented.

The major modifications in this change include:

- The type and names of the crc32c and md5_hash fields changed.
- Previously the md5 hash was sometimes represented as a hex-encoded
  binary blob and sometimes as just a binary blob. Now it is always
  a binary blob.
- Several fields (like `if_generation_match` and `crc32c`) are now
  marked as `optional` in the protos, instead of using `*Value`
  message wrappers.
- The proto version of `ObjectAccessControl` does not contain the
  `bucket`, `object` or `generation` fields.
- Likewise, `BucketAccessControl` does not contain the bucket name.
- We use https://github.com/googleapis/storage-testbench as it
  supports the v2/ protos.
@product-auto-label product-auto-label Bot added the api: storage Issues related to the Cloud Storage API. label Sep 2, 2021
@google-cla google-cla Bot added the cla: yes This human has signed the Contributor License Agreement. label Sep 2, 2021
@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: 85d212be85057eb105fb207dc8a70e7e5b8cba15

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: f680e8f6f5ae898dc0cda470cd2d759d85a5164f

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: c150b1c5bf08057e7aa2c6bebbc8058c24d556ac

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@codecov
Copy link
Copy Markdown

codecov Bot commented Sep 2, 2021

Codecov Report

Merging #7254 (7099a81) into main (cdbfdea) will decrease coverage by 0.01%.
The diff coverage is 85.54%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #7254      +/-   ##
==========================================
- Coverage   94.35%   94.34%   -0.02%     
==========================================
  Files        1317     1317              
  Lines      114502   114528      +26     
==========================================
+ Hits       108044   108048       +4     
- Misses       6458     6480      +22     
Impacted Files Coverage Δ
.../cloud/storage/benchmarks/throughput_experiment.cc 74.87% <0.00%> (+0.50%) ⬆️
google/cloud/storage/internal/grpc_client.h 100.00% <ø> (ø)
...e/cloud/storage/internal/grpc_object_read_source.h 100.00% <ø> (ø)
google/cloud/storage/internal/storage_auth.cc 100.00% <ø> (ø)
google/cloud/storage/internal/storage_auth.h 100.00% <ø> (ø)
...ogle/cloud/storage/internal/storage_round_robin.cc 100.00% <ø> (ø)
...oogle/cloud/storage/internal/storage_round_robin.h 100.00% <ø> (ø)
google/cloud/storage/internal/storage_stub.h 100.00% <ø> (ø)
...torage/tests/object_basic_crud_integration_test.cc 81.87% <0.00%> (-9.10%) ⬇️
google/cloud/storage/internal/grpc_client.cc 91.69% <93.22%> (-0.78%) ⬇️
... and 23 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cdbfdea...7099a81. Read the comment docs.

@coryan coryan changed the title feat(storage): move GCS+gRPC to v2/ protos feat(storage)!: move GCS+gRPC to v2/ protos Sep 2, 2021
@coryan coryan marked this pull request as ready for review September 2, 2021 03:35
@coryan coryan requested a review from a team September 2, 2021 03:35
@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: 7099a811b66c47790061eb4712ffaae726f91041

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

Copy link
Copy Markdown
Contributor

@devjgm devjgm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to see the emulator code now living in its own repo.

@coryan coryan merged commit 4f91fc0 into googleapis:main Sep 2, 2021
@coryan coryan deleted the feat-gcs-grpc-move-implementation-to-v2-pr1-no-emulator branch September 2, 2021 12:46
dbolduc pushed a commit to dbolduc/google-cloud-cpp that referenced this pull request Sep 4, 2021
GCS+gRPC has published a new set of protos to target. These protos
rename and change the types for a number of fields, and they *only*
describe the RPCs that are actually implemented.

The major modifications in this change include:

- The type and names of the crc32c and md5_hash fields changed.
- Previously the md5 hash was sometimes represented as a hex-encoded
  binary blob and sometimes as just a binary blob. Now it is always
  a binary blob.
- Several fields (like `if_generation_match` and `crc32c`) are now
  marked as `optional` in the protos, instead of using `*Value`
  message wrappers.
- The proto version of `ObjectAccessControl` does not contain the
  `bucket`, `object` or `generation` fields.
- Likewise, `BucketAccessControl` does not contain the bucket name.
- We use https://github.com/googleapis/storage-testbench as it
  supports the v2/ protos.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the Cloud Storage API. cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consider moving google/cloud/storage/emulator/... to a separate repo [Flake] //google/cloud/storage/emulator:test_utils fails

3 participants