Skip to content

Commit ff2866f

Browse files
author
Ajay Kannan
committed
---
yaml --- r: 953 b: refs/heads/master c: 3751460 h: refs/heads/master i: 951: f31bde7 v: v3
1 parent e7b6d8e commit ff2866f

19 files changed

Lines changed: 171 additions & 1002 deletions

File tree

[refs]

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
---
2-
refs/heads/master: 476d5942187ae1f89b7a020543f2104298a30bd6
2+
refs/heads/master: 3751460021aeba8a2043b709773c9436340a6947
33
refs/heads/travis: 0fa997e2fc9c6b61b2d91e6d163655aae67d44b6
44
refs/heads/gh-pages: 5a10432ecc75f29812e33a8236c900379509fe99

trunk/.travis.yml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,7 @@ before_install:
88
- git clone -b travis `git config --get remote.origin.url` target/travis
99
- cp target/travis/settings.xml ~/.m2/settings.xml
1010
install: mvn install -DskipTests=true -Dgpg.skip=true
11-
script:
12-
- utilities/verify.sh
11+
script: mvn verify
1312
branches:
1413
only:
1514
- master

trunk/README.md

Lines changed: 1 addition & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ Java idiomatic client for [Google Cloud Platform][cloud-platform] services.
1414
This client supports the following Google Cloud Platform services:
1515

1616
- [Google Cloud Datastore] (#google-cloud-datastore)
17-
- [Google Cloud Storage] (#google-cloud-storage)
1817

1918
<!---
2019
- [Google Cloud Storage] (https://cloud.google.com/storage/)
@@ -45,7 +44,7 @@ Example Applications
4544
Google Cloud Datastore
4645
----------------------
4746

48-
[Google Cloud Datastore][cloud-datastore] is a fully managed, schemaless database for
47+
Google [Cloud Datastore][cloud-datastore] is a fully managed, schemaless database for
4948
storing non-relational data. Cloud Datastore automatically scales with
5049
your users and supports ACID transactions, high availability of reads and
5150
writes, strong consistency for reads and ancestor queries, and eventual
@@ -87,46 +86,6 @@ if (entity == null) {
8786
}
8887
```
8988

90-
Google Cloud Storage
91-
----------------------
92-
93-
[Google Cloud Storage][cloud-storage] is a durable and highly available
94-
object storage service. Google Cloud Storage is almost infinitely scalable
95-
and guarantees consistency: when a write succeeds, the latest copy of the
96-
object will be returned to any GET, globally.
97-
98-
See the [Google Cloud Storage docs][cloud-storage-activation] for more details on how to activate
99-
Cloud Storage for your project.
100-
101-
See the ``gcloud-java`` API [storage documentation][storage-api] to learn how to interact
102-
with the Cloud Storage using this Client Library.
103-
104-
```java
105-
import static java.nio.charset.StandardCharsets.UTF_8;
106-
107-
import com.google.gcloud.storage.Blob;
108-
import com.google.gcloud.storage.Storage;
109-
import com.google.gcloud.storage.StorageFactory;
110-
import com.google.gcloud.storage.StorageOptions;
111-
112-
import java.nio.ByteBuffer;
113-
import java.nio.channels.WritableByteChannel;
114-
115-
StorageOptions options = StorageOptions.builder().projectId(PROJECT_ID).build();
116-
Storage storage = StorageFactory.instance().get(options);
117-
Blob blob = new Blob(storage, "bucket", "blob_name");
118-
if (!blob.exists()) {
119-
storage2.create(blob.info(), "Hello, Cloud Storage!".getBytes(UTF_8));
120-
} else {
121-
System.out.println("Updating content for " + blob.info().name());
122-
byte[] prevContent = blob.content();
123-
System.out.println(new String(prevContent, UTF_8));
124-
WritableByteChannel channel = blob.writer();
125-
channel.write(ByteBuffer.wrap("Updated content".getBytes(UTF_8)));
126-
channel.close();
127-
}
128-
```
129-
13089
Contributing
13190
------------
13291

@@ -171,5 +130,3 @@ Apache 2.0 - See [LICENSE] for more information.
171130
[cloud-storage]: https://cloud.google.com/storage/
172131
[cloud-storage-docs]: https://cloud.google.com/storage/docs/overview
173132
[cloud-storage-create-bucket]: https://cloud.google.com/storage/docs/cloud-console#_creatingbuckets
174-
[cloud-storage-activation]: https://cloud.google.com/storage/docs/signup
175-
[storage-api]: http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/storage/package-summary.html

trunk/TESTING.md

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
## gcloud-java tools for testing
2+
3+
gcloud-java provides tools to make testing your application easier.
4+
5+
### Testing interactions with Datastore
6+
7+
#### On your machine
8+
9+
You can test against a temporary local datastore by following these steps:
10+
11+
1. Start the local datastore emulator using `LocalGcdHelper`. This can be done in two ways:
12+
- Run `LocalGcdHelper.java`'s `main` method with arguments `START` and (optionally) `--port=<port number>`. This will create a temporary folder on your computer and bind `localhost:<port number>` for communication with the local datastore. The port number is an optional argument. If no port number is specified, port 8080 will be used.
13+
- Call `LocalGcdHelper.start(<project ID>, <port number>)` before running your tests. Save the `LocalGcdHelper` object returned so that you can stop the emulator later.
14+
15+
2. In your program, create and use a datastore whose host is set host to `localhost:<port number>`. For example,
16+
```java
17+
DatastoreOptions options = DatastoreOptions.builder()
18+
.projectId(PROJECT_ID)
19+
.host("localhost:8080")
20+
.build();
21+
Datastore localDatastore = DatastoreFactory.instance().get(options);
22+
```
23+
3. Run your tests.
24+
25+
4. Stop the local datastore emulator.
26+
- If you ran `LocalGcdHelper.java`'s `main` function to start the emulator, run `LocalGcdHelper.java`'s `main` method with arguments `STOP` and (optionally) `--port=<port number>`. If the port is not supplied, the program will attempt to close the last port started.
27+
- If you ran `LocalGcdHelper.start()` to start the emulator, call the `stop()` method on the `LocalGcdHelper` object returned by `LocalGcdHelper.start()`.
28+
29+
#### On a remote machine
30+
31+
You can test against a remote datastore emulator as well. To do this, set the `DatastoreOptions` project endpoint to the hostname of the remote machine, like the example below.
32+
33+
```java
34+
DatastoreOptions options = DatastoreOptions.builder()
35+
.projectId(PROJECT_ID)
36+
.host("http://<hostname of machine>")
37+
.build();
38+
Datastore localDatastore = DatastoreFactory.instance().get(options);
39+
```
40+
41+
Note that the remote datastore must be running before your tests are run. Also note that the `host` argument must start with "http://" or "https://".
42+
43+
44+
### Testing interactions with Storage
45+
46+
There currently isn't an emulator for Google Cloud Storage, so an alternative is to create a test project. `RemoteGcsHelper` contains convenience methods to make setting up and cleaning up the test project easier. To use this class, follow the steps below:
47+
48+
1. Create a test Google App Engine project.
49+
50+
2. Create and download a JSON key by going to the Google Developer's Console and clicking API's & Auth > Credentials > Add Credentials > Service Credentials. Choose "JSON", download the file, and note its location.
51+
52+
3. Set environment variables `GCLOUD_TESTS_PROJECT_ID` and `GCLOUD_TESTS_KEY_PATH` according to your test project's ID and the location of the newly-downloaded JSON key file.
53+
54+
4. Create and and use a `RemoteGcsHelper` object.
55+
Here is an example that uses the RemoteGcsHelper to create a bucket and clear the bucket at the end of the test.
56+
```java
57+
RemoteGcsHelper gcsHelper = RemoteGcsHelper.create();
58+
Storage storage = StorageFactory.instance().get(gcsHelper.options());
59+
String bucket = RemoteGcsHelper.generateBucketName();
60+
storage.create(BucketInfo.of(bucket));
61+
// Do tests using Storage
62+
RemoteGcsHelper.deleteBucketRecursively(storage, bucket, 5, TimeUnit.SECONDS);
63+
```

trunk/gcloud-java-datastore/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,13 @@ if (entity == null) {
6969
}
7070
```
7171

72+
Testing
73+
-------
74+
75+
This library makes testing interactions with Cloud Datastore easier by using a local datastore emulator.
76+
77+
See [TESTING] to read more about testing locally.
78+
7279
Contributing
7380
------------
7481

@@ -98,6 +105,7 @@ Apache 2.0 - See [LICENSE] for more information.
98105

99106
[CONTRIBUTING]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CONTRIBUTING.md
100107
[LICENSE]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/LICENSE
108+
[TESTING]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/TESTING.md
101109
[cloud-platform]: https://cloud.google.com/
102110
[cloud-datastore]: https://cloud.google.com/datastore/docs
103111
[cloud-datastore-docs]: https://cloud.google.com/datastore/docs

trunk/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/testing/package-info.java

Lines changed: 0 additions & 39 deletions
This file was deleted.

trunk/gcloud-java-datastore/src/main/java/com/google/gcloud/spi/DefaultDatastoreRpc.java

Lines changed: 1 addition & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,6 @@
3232
import com.google.api.services.datastore.client.DatastoreException;
3333
import com.google.api.services.datastore.client.DatastoreFactory;
3434
import com.google.api.services.datastore.client.DatastoreOptions.Builder;
35-
import com.google.common.base.Preconditions;
3635
import com.google.common.collect.ImmutableMap;
3736
import com.google.gcloud.datastore.DatastoreOptions;
3837
import com.google.gcloud.spi.DatastoreRpc.DatastoreRpcException.Reason;
@@ -41,10 +40,6 @@
4140
import org.json.JSONObject;
4241
import org.json.JSONTokener;
4342

44-
import java.net.InetAddress;
45-
import java.net.MalformedURLException;
46-
import java.net.URL;
47-
import java.net.UnknownHostException;
4843
import java.util.HashMap;
4944
import java.util.Map;
5045

@@ -67,45 +62,14 @@ public class DefaultDatastoreRpc implements DatastoreRpc {
6762
}
6863

6964
public DefaultDatastoreRpc(DatastoreOptions options) {
70-
String normalizedHost = normalizeHost(options.host());
7165
client = DatastoreFactory.get().create(
7266
new Builder()
7367
.dataset(options.projectId())
74-
.host(normalizedHost)
68+
.host(options.host())
7569
.initializer(options.httpRequestInitializer())
7670
.build());
7771
}
7872

79-
private static String normalizeHost(String host) {
80-
host = host.toLowerCase();
81-
if (includesScheme(host)) {
82-
Preconditions.checkArgument(!(host.startsWith("https://") && isLocalHost(host)),
83-
"\"https\" is not supported for localhost. Use \"http\" instead.");
84-
return host;
85-
}
86-
return "http://" + host;
87-
}
88-
89-
private static boolean isLocalHost(String host) {
90-
if (host != null) {
91-
try {
92-
String normalizedHost = host;
93-
if (!includesScheme(normalizedHost)) {
94-
normalizedHost = "http://" + normalizedHost;
95-
}
96-
InetAddress hostAddr = InetAddress.getByName(new URL(normalizedHost).getHost());
97-
return hostAddr.isAnyLocalAddress() || hostAddr.isLoopbackAddress();
98-
} catch (UnknownHostException | MalformedURLException e) {
99-
// ignore
100-
}
101-
}
102-
return false;
103-
}
104-
105-
private static boolean includesScheme(String url) {
106-
return url.startsWith("http://") || url.startsWith("https://");
107-
}
108-
10973
private static DatastoreRpcException translate(DatastoreException exception) {
11074
String message = exception.getMessage();
11175
String reasonStr = "";

trunk/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreOptionsTest.java

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,6 @@
2222
import static org.junit.Assert.assertSame;
2323
import static org.junit.Assert.assertTrue;
2424

25-
import com.google.gcloud.datastore.testing.LocalGcdHelper;
2625
import com.google.gcloud.spi.DatastoreRpc;
2726
import com.google.gcloud.spi.DatastoreRpcFactory;
2827

trunk/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreTest.java

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,6 @@
3333
import com.google.gcloud.datastore.StructuredQuery.OrderBy;
3434
import com.google.gcloud.datastore.StructuredQuery.Projection;
3535
import com.google.gcloud.datastore.StructuredQuery.PropertyFilter;
36-
import com.google.gcloud.datastore.testing.LocalGcdHelper;
3736
import com.google.gcloud.spi.DatastoreRpc;
3837
import com.google.gcloud.spi.DatastoreRpc.DatastoreRpcException.Reason;
3938
import com.google.gcloud.spi.DatastoreRpcFactory;

trunk/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/testing/LocalGcdHelper.java renamed to trunk/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/LocalGcdHelper.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
* limitations under the License.
1515
*/
1616

17-
package com.google.gcloud.datastore.testing;
17+
package com.google.gcloud.datastore;
1818

1919
import static java.nio.charset.StandardCharsets.UTF_8;
2020

0 commit comments

Comments
 (0)