Skip to content

feat(GCS+gRPC): implement standard parameters#7272

Merged
coryan merged 5 commits intogoogleapis:mainfrom
coryan:feat-gcs-grpc-implement-standard-query-parameters
Sep 8, 2021
Merged

feat(GCS+gRPC): implement standard parameters#7272
coryan merged 5 commits intogoogleapis:mainfrom
coryan:feat-gcs-grpc-implement-standard-query-parameters

Conversation

@coryan
Copy link
Copy Markdown
Contributor

@coryan coryan commented Sep 3, 2021

The REST API has a small number of "standard" query parameters which are
not part of the gRPC requests. They are implemented using special
x-goog-* "metadata" via the grpc::ClientContext. These are
documented at:

https://cloud.google.com/apis/docs/system-parameters

With this change the GCS+gRPC plugin fills out those headers from the
*Request object.

Fixes #4215 and fixes #6982


This change is Reviewable

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Sep 3, 2021
@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: 1b0822761a6be1b513ca2c852ec51fe306b2fb5c

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@codecov
Copy link
Copy Markdown

codecov Bot commented Sep 3, 2021

Codecov Report

Merging #7272 (bf31602) into main (0c6ccc8) will decrease coverage by 0.00%.
The diff coverage is 97.63%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #7272      +/-   ##
==========================================
- Coverage   93.79%   93.79%   -0.01%     
==========================================
  Files        1339     1341       +2     
  Lines      115461   115550      +89     
==========================================
+ Hits       108294   108376      +82     
- Misses       7167     7174       +7     
Impacted Files Coverage Δ
...oogle/cloud/storage/tests/grpc_integration_test.cc 97.76% <90.00%> (-2.24%) ⬇️
google/cloud/storage/internal/grpc_client.cc 92.49% <100.00%> (+0.39%) ⬆️
...d/storage/internal/grpc_configure_client_context.h 100.00% <100.00%> (ø)
...age/internal/grpc_configure_client_context_test.cc 100.00% <100.00%> (ø)
.../storage/internal/grpc_resumable_upload_session.cc 100.00% <100.00%> (ø)
google/cloud/testing_util/validate_metadata.cc 78.83% <100.00%> (ø)
google/cloud/storage/internal/curl_handle.h 80.00% <0.00%> (-2.86%) ⬇️
.../cloud/storage/benchmarks/throughput_experiment.cc 74.37% <0.00%> (-0.51%) ⬇️
...cloud/pubsub/internal/subscription_session_test.cc 97.78% <0.00%> (-0.25%) ⬇️
google/cloud/pubsub/samples/samples.cc 91.67% <0.00%> (-0.24%) ⬇️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0c6ccc8...bf31602. Read the comment docs.

@coryan coryan marked this pull request as ready for review September 3, 2021 22:20
@coryan coryan requested a review from a team September 3, 2021 22:20
@coryan
Copy link
Copy Markdown
Contributor Author

coryan commented Sep 3, 2021

/FYI: @tritone @noahdietz I am not sure if you implemented the quotaUser, userIp, or the fields query parameters for REST, but here is how I implemented them for gRPC.

Copy link
Copy Markdown
Member

@dbolduc dbolduc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers, but I have some questions

Reviewed 9 of 9 files at r1, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @coryan)


google/cloud/storage/internal/grpc_client.cc, line 270 at r1 (raw file):

  // extra prefix to ApplyQueryParameters sends the right filtering instructions
  // to the gRPC API.
  ApplyQueryParameters(*context, request, "resource");

I guess I don't understand why this is the only call that has the "resource" prefix.

e.g. the proto for QueryWriteStatusResponse has an Object resource.


google/cloud/storage/internal/grpc_configure_client_context.h, line 39 at r1 (raw file):

template <typename Request>
void ApplyQueryParameters(grpc::ClientContext& context, Request const& request,
                          std::string const& prefix = std::string{}) {

is the prefix ever anything other than "" or "resource"? Not saying we should, but it could be something like bool use_resource = false


google/cloud/storage/internal/grpc_configure_client_context.h, line 57 at r1 (raw file):

  if (request.template HasOption<Fields>()) {
    auto field_mask = request.template GetOption<Fields>().value();
    if (!prefix.empty()) field_mask = prefix + "(" + field_mask + ")";

This looks nicer, but absl::StrCat() is possible.


google/cloud/storage/internal/grpc_configure_client_context_test.cc, line 1 at r1 (raw file):

// Copyright 2019 Google LLC

2021

The REST API has a small number of "standard" query parameters which are
not part of the gRPC requests. They are implemented using special
`x-goog-*` "metadata" via the `grpc::ClientContext`. These are
documented at:

https://cloud.google.com/apis/docs/system-parameters

With this change the GCS+gRPC plugin fills out those headers from the
`*Request` object.
@coryan coryan force-pushed the feat-gcs-grpc-implement-standard-query-parameters branch from 1b08227 to 6c19f4d Compare September 8, 2021 16:42
@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: 6c19f4d7356b1726885efb385113af0a32be65cd

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

Copy link
Copy Markdown
Contributor Author

@coryan coryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL.

Reviewable status: 5 of 10 files reviewed, all discussions resolved (waiting on @dbolduc)


google/cloud/storage/internal/grpc_client.cc, line 270 at r1 (raw file):

Previously, dbolduc (Darren Bolduc) wrote…

I guess I don't understand why this is the only call that has the "resource" prefix.

e.g. the proto for QueryWriteStatusResponse has an Object resource.

Good question, fixed here and elsewhere. Actually I had to create a few additional PRs to fix this for REST too.


google/cloud/storage/internal/grpc_configure_client_context.h, line 39 at r1 (raw file):

Previously, dbolduc (Darren Bolduc) wrote…

is the prefix ever anything other than "" or "resource"? Not saying we should, but it could be something like bool use_resource = false

For now that is it, but I expect that might change as we implement more APIs over gRPC.


google/cloud/storage/internal/grpc_configure_client_context.h, line 57 at r1 (raw file):

Previously, dbolduc (Darren Bolduc) wrote…

This looks nicer, but absl::StrCat() is possible.

I am leaving as-is, it is readable enough for me.


google/cloud/storage/internal/grpc_configure_client_context_test.cc, line 1 at r1 (raw file):

Previously, dbolduc (Darren Bolduc) wrote…

2021

Fixed.

@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: c044167d5eaef8fdd2164e34ce6e70648c229c81

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: bf31602641db280367e5f0f33b956d5e7ec65f2f

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@coryan coryan merged commit db914e2 into googleapis:main Sep 8, 2021
@coryan coryan deleted the feat-gcs-grpc-implement-standard-query-parameters branch September 8, 2021 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move GCS+gRPC plugin to v2 protos Handle storage::Fields in GCS+gRPC plugin

5 participants