Skip to content

Commit 80aa90e

Browse files
igorbernstein2garrettjonesgoogle
igorbernstein2
authored and
garrettjonesgoogle
committed
Bigtable: 01. Surface: Base structure (#2842)
Layout the base structure to overlay the Bigtable GAPIC client. The general idea is for EnhancedBigtableStub to wrap the GAPIC GrpcBigtableStub and for it to construct Callable chains on top of the generated chains.
1 parent a7b9048 commit 80aa90e

9 files changed

Lines changed: 757 additions & 7 deletions

File tree

‎google-cloud-bigtable/DEVELOPING.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Developing
2+
3+
## Overview
4+
5+
The Cloud Bigtable client is composed of two layers:
6+
7+
* an automatically generated GAPIC base layer
8+
* a handwritten layer
9+
10+
11+
### Generated GAPIC base layer
12+
13+
The GAPIC layer is generated from a combination of protobuf and GAPIC yaml definitions found in
14+
[googleapis](https://github.com/googleapis/googleapis/tree/master/google/bigtable). This layer is
15+
split into 2 parts: admin and data apis. The admin part is straightforward enough to be consumed
16+
directly by the consumer. However the data client is too protocol focused and is too low level to
17+
be consumed by an application directly. Instead, it's intended to be an implementation detail of the
18+
handwritten layer.
19+
20+
21+
### Handwritten layer
22+
23+
The handwritten layer for the data clients is meant to wrap the GAPIC layer entirely. This is
24+
necessary because some APIs are too protocol focused (ReadRows chunks vs logical row). Furthermore
25+
by constraining the api, the surface api can be made more ergonomic:
26+
27+
* method calls have an implicit context of an instance
28+
* timestamps can be generated on the client side to make mutations idempotent
29+
* a handwritten api is a lot less verbose than the autogenerated protobuf api
30+
31+
32+
## Structure
33+
34+
* BigtableDataSettings: replaces BaseBigtableDataSettings. It's a simple wrapper of
35+
stub/EnhancedBigtableStubSettings. This is meant to be the public facing api of settings.
36+
* BigtableDataClient: replaces BaseBigtableDataClient. It provides an ergonomic wrapper around
37+
stub/EnhancedBigtableStub. This class is meant to be simple syntactic sugar around the stub.
38+
* stub/EnhancedBigtableStubSettings: wraps the autogenerated stub/BigtableStubSettings. This class
39+
exposes all of the necessary settings. The settings can be categorized as:
40+
* General settings that affect all RPCs and include things like the target instance and connection
41+
pooling
42+
* Method specific settings that allow the user to tune each RPC individually
43+
* stub/EnhancedBigtableStub: wraps the autogenerated GrpcBigtableStub. It layers logical structure
44+
on top of the protocol level requests and responses. It is responsible for adapting ReadRows
45+
chunks to logical rows and for converting between raw protobufs and user facing wrappers.
46+
* wrappers/: contains user facing wrappers for the protobufs. Each wrapper has a toProto method
47+
that will return the corresponding proto.
48+
* internal/: contains internal implementation details. The end user doesn't directly interact with
49+
these classes. It contains things like RequestContext, that is used by all toProto methods to
50+
contextualize the user facing wrapper with the target instance and app profile.

‎google-cloud-bigtable/pom.xml‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,14 @@
5656
<groupId>io.grpc</groupId>
5757
<artifactId>grpc-auth</artifactId>
5858
</dependency>
59+
60+
<dependency>
61+
<groupId>com.google.auto.value</groupId>
62+
<artifactId>auto-value</artifactId>
63+
<scope>provided</scope>
64+
</dependency>
65+
66+
<!-- Test -->
5967
<dependency>
6068
<groupId>${project.groupId}</groupId>
6169
<artifactId>google-cloud-core</artifactId>
@@ -68,13 +76,8 @@
6876
<scope>test</scope>
6977
</dependency>
7078
<dependency>
71-
<groupId>org.easymock</groupId>
72-
<artifactId>easymock</artifactId>
73-
<scope>test</scope>
74-
</dependency>
75-
<dependency>
76-
<groupId>org.objenesis</groupId>
77-
<artifactId>objenesis</artifactId>
79+
<groupId>org.mockito</groupId>
80+
<artifactId>mockito-all</artifactId>
7881
<scope>test</scope>
7982
</dependency>
8083
<dependency>

‎google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/BigtableDataClient.java‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
/*
2+
* Copyright 2018 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* https://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
package com.google.cloud.bigtable.data.v2;
17+
18+
import com.google.api.core.InternalApi;
19+
import com.google.bigtable.admin.v2.InstanceName;
20+
import com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStub;
21+
import java.io.IOException;
22+
23+
/**
24+
* Client for reading from and writing to existing Bigtable tables.
25+
*
26+
* <p>This class provides the ability to make remote calls to the backing service. Sample code to
27+
* get started:
28+
*
29+
* <pre>{@code
30+
* InstanceName instanceName = InstanceName.of("[PROJECT]", "[INSTANCE]");
31+
* try (BigtableDataClient bigtableDataClient = BigtableDataClient.create(instanceName)) {
32+
* // TODO: add example usage
33+
* }
34+
* }</pre>
35+
*
36+
* <p>Note: close() needs to be called on the bigtableDataClient object to clean up resources such
37+
* as threads. In the example above, try-with-resources is used, which automatically calls close().
38+
*
39+
* <p>The surface of this class includes several types of Java methods for each of the API's
40+
* methods:
41+
*
42+
* <ol>
43+
* <li>A "flattened" method. With this type of method, the fields of the request type have been
44+
* converted into function parameters. It may be the case that not all fields are available as
45+
* parameters, and not every API method will have a flattened method entry point.
46+
* <li>A "callable" method. This type of method takes no parameters and returns an immutable API
47+
* callable object, which can be used to initiate calls to the service.
48+
* </ol>
49+
*
50+
* <p>See the individual methods for example code.
51+
*
52+
* <p>This class can be customized by passing in a custom instance of BigtableDataSettings to
53+
* create(). For example:
54+
*
55+
* <p>To customize credentials:
56+
*
57+
* <pre>{@code
58+
* BigtableDataSettings bigtableDataSettings =
59+
* BigtableDataSettings.newBuilder()
60+
* .setInstanceName(InstanceName.of("[PROJECT]", "[INSTANCE]"))
61+
* .setCredentialsProvider(FixedCredentialsProvider.create(myCredentials))
62+
* .build();
63+
* try(BigtableDataClient bigtableDataClient = BigtableDataClient.create(bigtableDataSettings)) {
64+
* // ..
65+
* }
66+
* }</pre>
67+
*
68+
* To customize the endpoint:
69+
*
70+
* <pre>{@code
71+
* BigtableDataSettings bigtableDataSettings =
72+
* BigtableDataSettings.newBuilder()
73+
* .setInstanceName(InstanceName.of("[PROJECT]", "[INSTANCE]"))
74+
* .setEndpoint(myEndpoint).build();
75+
* try(BigtableDataClient bigtableDataClient = BigtableDataClient.create(bigtableDataSettings)) {
76+
* // ..
77+
* }
78+
* }</pre>
79+
*/
80+
public class BigtableDataClient implements AutoCloseable {
81+
private final EnhancedBigtableStub stub;
82+
83+
/**
84+
* Constructs an instance of BigtableClient with default settings.
85+
*
86+
* @param instanceName The instance to connect to.
87+
* @return A new client.
88+
* @throws IOException If any.
89+
*/
90+
public static BigtableDataClient create(InstanceName instanceName) throws IOException {
91+
BigtableDataSettings settings =
92+
BigtableDataSettings.newBuilder().setInstanceName(instanceName).build();
93+
return create(settings);
94+
}
95+
96+
/**
97+
* Constructs an instance of BigtableDataClient, using the given settings. The channels are
98+
* created based on the settings passed in, or defaults for any settings that are not set.
99+
*/
100+
public static BigtableDataClient create(BigtableDataSettings settings) throws IOException {
101+
EnhancedBigtableStub stub = EnhancedBigtableStub.create(settings.getTypedStubSettings());
102+
return new BigtableDataClient(stub);
103+
}
104+
105+
@InternalApi("Visible for testing")
106+
BigtableDataClient(EnhancedBigtableStub stub) {
107+
this.stub = stub;
108+
}
109+
110+
/** Close the clients and releases all associated resources. */
111+
@Override
112+
public void close() throws Exception {
113+
stub.close();
114+
}
115+
}

‎google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/BigtableDataSettings.java‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
/*
2+
* Copyright 2018 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* https://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
package com.google.cloud.bigtable.data.v2;
17+
18+
import com.google.api.gax.rpc.ClientSettings;
19+
import com.google.bigtable.admin.v2.InstanceName;
20+
import com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStubSettings;
21+
import java.io.IOException;
22+
import javax.annotation.Nonnull;
23+
24+
/**
25+
* Settings class to configure an instance of {@link BigtableDataClient}.
26+
*
27+
* <p>Sane defaults are provided for most settings:
28+
*
29+
* <ul>
30+
* <li>The default service address (bigtable.googleapis.com) and default port (443) are used.
31+
* <li>Credentials are acquired automatically through Application Default Credentials.
32+
* <li>Retries are configured for idempotent methods but not for non-idempotent methods.
33+
* </ul>
34+
*
35+
* <p>The only required setting is the instance name.
36+
*
37+
* <p>The builder of this class is recursive, so contained classes are themselves builders. When
38+
* build() is called, the tree of builders is called to create the complete settings object.
39+
*
40+
* <pre>{@code
41+
* BigtableDataSettings bigtableDataSettings = BigtableDataSettings.newBuilder()
42+
* .setInstanceName(InstanceName.of("my-project", "my-instance-id"))
43+
* .setAppProfileId("default")
44+
* .build();
45+
* }</pre>
46+
*/
47+
public class BigtableDataSettings extends ClientSettings<BigtableDataSettings> {
48+
private BigtableDataSettings(Builder builder) throws IOException {
49+
super(builder);
50+
}
51+
52+
/** Create a new builder. */
53+
public static Builder newBuilder() {
54+
return new Builder();
55+
}
56+
57+
/** Returns the target instance */
58+
public InstanceName getInstanceName() {
59+
return getTypedStubSettings().getInstanceName();
60+
}
61+
62+
/** Returns the configured AppProfile to use */
63+
public String getAppProfileId() {
64+
return getTypedStubSettings().getAppProfileId();
65+
}
66+
67+
@SuppressWarnings("unchecked")
68+
EnhancedBigtableStubSettings getTypedStubSettings() {
69+
return (EnhancedBigtableStubSettings) getStubSettings();
70+
}
71+
72+
/** Returns a builder containing all the values of this settings class. */
73+
@SuppressWarnings("unchecked")
74+
public Builder toBuilder() {
75+
return new Builder(this);
76+
}
77+
78+
/** Builder for BigtableDataSettings. */
79+
public static class Builder extends ClientSettings.Builder<BigtableDataSettings, Builder> {
80+
/**
81+
* Initializes a new Builder with sane defaults for all settings.
82+
*
83+
* <p>Most defaults are extracted from BaseBigtableDataSettings, however some of the more
84+
* complex defaults are configured explicitly here. Once the overlayed defaults are configured,
85+
* the base settings are augmented to work with overlayed functionality (like disabling retries
86+
* in the underlying GAPIC client for batching).
87+
*/
88+
private Builder() {
89+
super(EnhancedBigtableStubSettings.newBuilder());
90+
}
91+
92+
private Builder(BigtableDataSettings settings) {
93+
super(settings);
94+
}
95+
96+
// <editor-fold desc="Public API">
97+
/**
98+
* Sets the target instance. This setting is required. All RPCs will be made in the context of
99+
* this setting.
100+
*/
101+
public Builder setInstanceName(@Nonnull InstanceName instanceName) {
102+
getTypedStubSettings().setInstanceName(instanceName);
103+
return this;
104+
}
105+
106+
/** Gets the {@link InstanceName} that was previously set on this Builder. */
107+
public InstanceName getInstanceName() {
108+
return getTypedStubSettings().getInstanceName();
109+
}
110+
111+
/**
112+
* Sets the AppProfile to use. An application profile (sometimes also shortened to "app
113+
* profile") is a group of configuration parameters for an individual use case. A client will
114+
* identify itself with an application profile ID at connection time, and the requests will be
115+
* handled according to that application profile.
116+
*/
117+
public Builder setAppProfileId(@Nonnull String appProfileId) {
118+
getTypedStubSettings().setAppProfileId(appProfileId);
119+
return this;
120+
}
121+
122+
/** Gets the app profile id that was previously set on this Builder. */
123+
public String getAppProfileId() {
124+
return getTypedStubSettings().getAppProfileId();
125+
}
126+
127+
@SuppressWarnings("unchecked")
128+
private EnhancedBigtableStubSettings.Builder getTypedStubSettings() {
129+
return (EnhancedBigtableStubSettings.Builder) getStubSettings();
130+
}
131+
132+
public BigtableDataSettings build() throws IOException {
133+
return new BigtableDataSettings(this);
134+
}
135+
// </editor-fold>
136+
}
137+
}

‎google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/internal/RequestContext.java‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
/*
2+
* Copyright 2018 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* https://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
package com.google.cloud.bigtable.data.v2.internal;
17+
18+
import com.google.api.core.InternalApi;
19+
import com.google.auto.value.AutoValue;
20+
import com.google.bigtable.admin.v2.InstanceName;
21+
22+
/**
23+
* Contains information necessary to construct Bigtable protobuf requests from user facing wrappers.
24+
*
25+
* <p>The intention is to extract repetitive details like instance names and app profiles into a
26+
* configurable values in {@link com.google.cloud.bigtable.data.v2.BigtableDataSettings} and expose
27+
* them (via this class) to each wrapper's toProto method.
28+
*
29+
* <p>This class is considered an internal implementation detail and not meant to be used by
30+
* applications.
31+
*/
32+
@InternalApi
33+
@AutoValue
34+
public abstract class RequestContext {
35+
public static RequestContext create(InstanceName instanceName, String appProfileId) {
36+
return new AutoValue_RequestContext(instanceName, appProfileId);
37+
}
38+
39+
/** The instance that the client is configured to target */
40+
public abstract InstanceName getInstanceName();
41+
42+
/** The App Profile to use when processing the current request */
43+
public abstract String getAppProfileId();
44+
}

0 commit comments

Comments
 (0)