Skip to content

Spanner: fix possible thread leak in Spanner.close()#5060

Merged
olavloite merged 7 commits intogoogleapis:masterfrom
olavloite:spanner-fix-leaked-threads
May 6, 2019
Merged

Spanner: fix possible thread leak in Spanner.close()#5060
olavloite merged 7 commits intogoogleapis:masterfrom
olavloite:spanner-fix-leaked-threads

Conversation

@olavloite
Copy link
Copy Markdown
Contributor

A Spanner instance holds a reference to three underlying gRPC stubs: spannerStub, instanceAdminStub and databaseAdminStub. These should all have their own thread pool for gRPC calls. This was achieved by setting an executor provider on the channel provider for the stubs. The worker threads of the thread pools created by this executor provider were however not shutdown when the stubs were closed, causing thread leaks if an application would open and close multiple Spanner instances during its lifetime.

Fixes #5059

A Spanner instance holds a reference to three underlying gRPC stubs:
spannerStub, instanceAdminStub and databaseAdminStub. These should all
have their own thread pool for gRPC calls. This was achieved by setting
an executor provider on the channel provider for the stubs. The worker
threads of the thread pools created by this executor provider were
however not shutdown when the stubs were closed, causing thread leaks if
an application would open and close multiple Spanner instances during
its lifetime.
@olavloite olavloite requested a review from a team May 4, 2019 06:21
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label May 4, 2019
@olavloite olavloite requested a review from kolea2 May 4, 2019 06:22
@codecov
Copy link
Copy Markdown

codecov Bot commented May 4, 2019

Codecov Report

Merging #5060 into master will decrease coverage by 0.3%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #5060      +/-   ##
============================================
- Coverage     50.38%   50.07%   -0.31%     
+ Complexity    23735    22379    -1356     
============================================
  Files          2248     2248              
  Lines        226478   221803    -4675     
  Branches      24954    24836     -118     
============================================
- Hits         114113   111075    -3038     
+ Misses       103772   103016     -756     
+ Partials       8593     7712     -881
Impacted Files Coverage Δ Complexity Δ
...m/google/cloud/vision/v1/ImageAnnotatorClient.java 58.69% <0%> (-3.31%) 15% <0%> (-4%)
...e/cloud/vision/v1p4beta1/ImageAnnotatorClient.java 58.69% <0%> (-3.31%) 15% <0%> (-4%)
...gle/cloud/storage/testing/RemoteStorageHelper.java 61.98% <0%> (-3.06%) 9% <0%> (ø)
...e/cloud/vision/v1p3beta1/ImageAnnotatorClient.java 47.05% <0%> (-2.95%) 9% <0%> (-2%)
...e/cloud/vision/v1p2beta1/ImageAnnotatorClient.java 47.05% <0%> (-2.95%) 9% <0%> (-2%)
.../java/com/google/cloud/speech/v1/SpeechClient.java 48.57% <0%> (-2.78%) 10% <0%> (-2%)
...om/google/cloud/speech/v1p1beta1/SpeechClient.java 48.57% <0%> (-2.78%) 10% <0%> (-2%)
...d/webrisk/v1beta1/WebRiskServiceV1Beta1Client.java 63.41% <0%> (-2.5%) 12% <0%> (-3%)
...ogle/cloud/texttospeech/v1/TextToSpeechClient.java 55.88% <0%> (-2.46%) 9% <0%> (-2%)
...cloud/texttospeech/v1beta1/TextToSpeechClient.java 55.88% <0%> (-2.46%) 9% <0%> (-2%)
... and 523 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 32551b2...fd72ccf. Read the comment docs.

olavloite added 4 commits May 4, 2019 21:45
The custom ThreadFactory is not necessary for this fix.
Setting an executor on the stub settings is not necessary, as the
default for these settings are all an InstantiatingExecutorProvider.

/** Implementation of Cloud Spanner remote calls using Gapic libraries. */
public class GapicSpannerRpc implements SpannerRpc {
// Thread factory to use to create our worker threads
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor - think you can remove this comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, this is no longer valid.

options.getInterceptorProvider(),
SpannerInterceptorProvider.createDefault()))
.setHeaderProvider(mergedHeaderProvider)
.setExecutorProvider(InstantiatingExecutorProvider.newBuilder().build())
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since each underlying grpc stub gets set this executor provider by default, is it correct that each stub will not use a shared executor for its transport channel (the one potential problem listed in the issue with this change)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct.

@olavloite olavloite merged commit 8d7b28f into googleapis:master May 6, 2019
@snehashah16 snehashah16 requested a review from yihanzhen May 6, 2019 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spanner: Spanner.close() does not clean up all resources

3 participants