Skip to content

feat(common): a generic long-running operation#6804

Merged
coryan merged 8 commits intogoogleapis:mainfrom
coryan:feat-implement-async-polling-loop-helpers
Jun 18, 2021
Merged

feat(common): a generic long-running operation#6804
coryan merged 8 commits intogoogleapis:mainfrom
coryan:feat-implement-async-polling-loop-helpers

Conversation

@coryan
Copy link
Copy Markdown
Contributor

@coryan coryan commented Jun 17, 2021

Long-running operations aip/151 are used for API methods that take a
significant amount of time to complete (think minutes, maybe an hour).
The gRPC API returns a "promise" object, represented by the
google::longrunning::Operation proto, and the application (or client
library) should periodically poll this object until it is "done".

In the C++ client libraries we represent these long-running operations
by a member function that returns future<StatusOr<ReturnType>>. This
PR introduces a generic function to implement these member functions. It
first starts the operation using an asynchronous retry loop, and then
starts an asynchronous loop to poll the operation until it completes.

The promise can complete with an error, which is represented by a
google::cloud::Status object, or with success and some ReturnType
value. The application may also configure the "polling policy", which
may stop the polling even though the operation has not completed.

A future PR will change the generated libraries to use this function.
With some work we could change the Spanner library to use it too,
without the need to launch detached threads to poll the results. With
substantial work we could change the Bigtable library to use it too.

Part of the work for #5934, for #4038, and for #6821


This change is Reviewable

Long-running operations [aip/151] are used for API methods that take a
significant amount of time to complete (think minutes, maybe an hour).
The gRPC API returns a "promise" object, represented by the
`google::longrunning::Operation` proto, and the application (or client
library) should periodically poll this object until it is "done".

In the C++ client libraries we represent these long-running operations
by a member function that returns `future<StatusOr<ReturnType>>`. This
PR introduces a generic function to implement these member functions. It
first starts the operation using an asynchronous retry loop, and then
starts an asynchronous loop to poll the operation until it completes.

The promise can complete with an error, which is represented by a
`google::cloud::Status` object, or with success and some `ReturnType`
value.  The application may also configure the "polling policy", which
may stop the polling even though the operation has not completed.

A future PR will change the generated libraries to use this function.
With some work we could change the Spanner library to use it too,
without the need to launch detached threads to poll the results. With
substantial work we could change the Bigtable library to use it too.

[aip/151]: https://google.aip.dev/151
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jun 17, 2021
@google-cla google-cla Bot added the cla: yes This human has signed the Contributor License Agreement. label Jun 17, 2021
@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: f507071c1ec603f051e7072d1b1690445c4b4d6c

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: 479f5f3bdeb0e74f44c680066d070209270ff196

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 18, 2021

Codecov Report

Merging #6804 (7a4cc32) into main (0bf4bb8) will increase coverage by 0.00%.
The diff coverage is 98.80%.

Impacted file tree graph

@@           Coverage Diff            @@
##             main    #6804    +/-   ##
========================================
  Coverage   95.50%   95.51%            
========================================
  Files        1257     1262     +5     
  Lines      112769   113105   +336     
========================================
+ Hits       107704   108032   +328     
- Misses       5065     5073     +8     
Impacted Files Coverage Δ
...ogle/cloud/internal/async_long_running_operation.h 95.65% <95.65%> (ø)
...loud/internal/async_long_running_operation_test.cc 96.47% <96.47%> (ø)
...gle/cloud/internal/async_long_running_operation.cc 100.00% <100.00%> (ø)
google/cloud/internal/async_polling_loop.cc 100.00% <100.00%> (ø)
google/cloud/internal/async_polling_loop_test.cc 100.00% <100.00%> (ø)
...e/cloud/spanner/testing/cleanup_stale_instances.cc 54.54% <0.00%> (-21.22%) ⬇️
google/cloud/storage/well_known_parameters.h 89.00% <0.00%> (-1.00%) ⬇️
google/cloud/spanner/internal/spanner_stub.cc 75.47% <0.00%> (-0.95%) ⬇️
...le/cloud/storage/internal/curl_download_request.cc 80.97% <0.00%> (-0.45%) ⬇️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0bf4bb8...7a4cc32. Read the comment docs.

@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: 3c694065dfe6c4b67bebd58577eed01223592dc3

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@coryan coryan marked this pull request as ready for review June 18, 2021 03:05
@coryan coryan requested a review from a team June 18, 2021 03:05
Comment thread google/cloud/internal/async_polling_loop.h Outdated
Comment thread google/cloud/internal/async_polling_loop.h Outdated
Comment thread google/cloud/internal/async_polling_loop.h Outdated
Comment thread google/cloud/internal/async_polling_loop.h Outdated
Comment thread google/cloud/internal/async_polling_loop.h Outdated
Comment thread google/cloud/internal/async_polling_loop_test.cc
Comment thread google/cloud/internal/async_long_running.h
Comment thread google/cloud/internal/async_long_running.h Outdated
Comment thread google/cloud/internal/async_long_running.h Outdated
Comment thread google/cloud/internal/async_long_running_test.cc Outdated
@coryan
Copy link
Copy Markdown
Contributor Author

coryan commented Jun 18, 2021

PTAL

@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: bfd3baa839e79a977c42305c386c5d6ac8fbec97

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

Copy link
Copy Markdown
Contributor

@devjgm devjgm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple comments/questions, but I don't think any of mine were blockers.

Comment thread google/cloud/internal/async_long_running.cc Outdated
Comment thread google/cloud/internal/async_long_running.h Outdated
Comment thread google/cloud/internal/async_long_running.h
@google-cloud-cpp-bot
Copy link
Copy Markdown
Contributor

Google Cloud Build Logs
For commit: 7a4cc32cd89b108e17494172a567b8ab74ade7f9

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@coryan coryan merged commit e9b7bff into googleapis:main Jun 18, 2021
@coryan coryan deleted the feat-implement-async-polling-loop-helpers branch June 18, 2021 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants