Skip to content

Commit fcfcefe

Browse files
author
Ajay Kannan
committed
---
yaml --- r: 2503 b: refs/heads/update-datastore c: 791fba1 h: refs/heads/master i: 2501: 9613ce0 2499: 9331ba1 2495: 88f21ea
1 parent 61e1050 commit fcfcefe

9 files changed

Lines changed: 57 additions & 80 deletions

File tree

‎[refs]‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ refs/heads/gh-pages: 4e0561bb4504bf647db669a14417b2b2c87ba45d
55
refs/heads/bigquery: 762fa5830e6c398c0396177e3e7fd243bd62cfc3
66
refs/heads/pubsub-alpha: 1a0e970f265af871e02274085b9662b3fe29058b
77
refs/heads/resource-manager: ebf4adc5ee835cd2086c4ac5b4e78d01a5a005a7
8-
refs/heads/update-datastore: dba9efb4cb6967659a920e2a8dfe7b2635c540b0
8+
refs/heads/update-datastore: 791fba190e2d076468451ece01936deea8262236
99
refs/tags/0.0.9: 22f1839238f66c39e67ed4dfdcd273b1ae2e8444
1010
refs/tags/v0.0.10: 207ebd2a3472fddee69fe1298eb90429e3306efd
1111
refs/tags/v0.0.11: ffbfba48a6426ff63c08ff2117e58681f251fbf2

‎branches/update-datastore/README.md‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,9 @@ Most `gcloud-java` libraries require a project ID. There are multiple ways to s
7575
Authentication
7676
--------------
7777

78-
There are multiple ways to authenticate to use Google Cloud services.
78+
First, ensure that the necessary Google Cloud APIs are enabled for your project. To do this, follow the instructions on the [authentication document](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/authentication) shared by all the gcloud language libraries.
79+
80+
Next, choose a method for authenticating API requests from within your project:
7981

8082
1. When using `gcloud-java` libraries from within Compute/App Engine, no additional authentication steps are necessary.
8183
2. When using `gcloud-java` libraries elsewhere, there are two options:
@@ -180,6 +182,11 @@ if (blob == null) {
180182
}
181183
```
182184
185+
Troubleshooting
186+
---------------
187+
188+
To get help, follow the `gcloud-java` links in the `gcloud-*` [shared Troubleshooting document](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/troubleshooting).
189+
183190
Java Versions
184191
-------------
185192
@@ -206,7 +213,7 @@ Contributing
206213
207214
Contributions to this library are always welcome and highly encouraged.
208215
209-
See [CONTRIBUTING] for more information on how to get started.
216+
See `gcloud-java`'s [CONTRIBUTING] documentation and the `gcloud-*` [shared documentation](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/contributing) for more information on how to get started.
210217
211218
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more information.
212219

‎branches/update-datastore/gcloud-java-core/README.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,11 @@ If you are using SBT, add this to your dependencies
2929
libraryDependencies += "com.google.gcloud" % "gcloud-java-core" % "0.0.12"
3030
```
3131

32+
Troubleshooting
33+
---------------
34+
35+
To get help, follow the `gcloud-java` links in the `gcloud-*` [shared Troubleshooting document](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/troubleshooting).
36+
3237
Java Versions
3338
-------------
3439

@@ -39,7 +44,9 @@ Contributing
3944

4045
Contributions to this library are always welcome and highly encouraged.
4146

42-
See [CONTRIBUTING] for more information on how to get started.
47+
See `gcloud-java`'s [CONTRIBUTING] documentation and the `gcloud-*` [shared documentation](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/contributing) for more information on how to get started.
48+
49+
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more information.
4350

4451
Versioning
4552
----------
@@ -57,5 +64,6 @@ Apache 2.0 - See [LICENSE] for more information.
5764

5865

5966
[CONTRIBUTING]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CONTRIBUTING.md
67+
[code-of-conduct]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CODE_OF_CONDUCT.md
6068
[LICENSE]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/LICENSE
6169
[cloud-platform]: https://cloud.google.com/

‎branches/update-datastore/gcloud-java-datastore/README.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,11 @@ public class GcloudJavaDatastoreExample {
189189
}
190190
```
191191

192+
Troubleshooting
193+
---------------
194+
195+
To get help, follow the `gcloud-java` links in the `gcloud-*` [shared Troubleshooting document](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/troubleshooting).
196+
192197
Java Versions
193198
-------------
194199

@@ -215,7 +220,9 @@ Contributing
215220

216221
Contributions to this library are always welcome and highly encouraged.
217222

218-
See [CONTRIBUTING] for more information on how to get started.
223+
See `gcloud-java`'s [CONTRIBUTING] documentation and the `gcloud-*` [shared documentation](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/contributing) for more information on how to get started.
224+
225+
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more information.
219226

220227
License
221228
-------
@@ -224,6 +231,7 @@ Apache 2.0 - See [LICENSE] for more information.
224231

225232

226233
[CONTRIBUTING]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CONTRIBUTING.md
234+
[code-of-conduct]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CODE_OF_CONDUCT.md
227235
[LICENSE]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/LICENSE
228236
[TESTING]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/TESTING.md#testing-code-that-uses-datastore
229237
[cloud-platform]: https://cloud.google.com/

‎branches/update-datastore/gcloud-java-examples/README.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,11 @@ To run examples from your command line:
5858
$mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.StorageExample" -Dexec.args="delete <bucket_name> test.txt"
5959
```
6060

61+
Troubleshooting
62+
---------------
63+
64+
To get help, follow the `gcloud-java` links in the `gcloud-*` [shared Troubleshooting document](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/troubleshooting).
65+
6166
Java Versions
6267
-------------
6368

@@ -77,7 +82,9 @@ Contributing
7782

7883
Contributions to this library are always welcome and highly encouraged.
7984

80-
See [CONTRIBUTING] for more information on how to get started.
85+
See `gcloud-java`'s [CONTRIBUTING] documentation and the `gcloud-*` [shared documentation](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/contributing) for more information on how to get started.
86+
87+
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more information.
8188

8289
License
8390
-------
@@ -86,6 +93,7 @@ Apache 2.0 - See [LICENSE] for more information.
8693

8794

8895
[CONTRIBUTING]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CONTRIBUTING.md
96+
[code-of-conduct]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CODE_OF_CONDUCT.md
8997
[LICENSE]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/LICENSE
9098
[cloud-platform]: https://cloud.google.com/
9199
[developers-console]:https://console.developers.google.com/

‎branches/update-datastore/gcloud-java-storage/README.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,11 @@ if (!blob.exists()) {
8282
}
8383
```
8484

85+
Troubleshooting
86+
---------------
87+
88+
To get help, follow the `gcloud-java` links in the `gcloud-*` [shared Troubleshooting document](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/troubleshooting).
89+
8590
Java Versions
8691
-------------
8792

@@ -108,7 +113,9 @@ Contributing
108113

109114
Contributions to this library are always welcome and highly encouraged.
110115

111-
See [CONTRIBUTING] for more information on how to get started.
116+
See `gcloud-java`'s [CONTRIBUTING] documentation and the `gcloud-*` [shared documentation](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/contributing) for more information on how to get started.
117+
118+
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more information.
112119

113120
License
114121
-------
@@ -117,6 +124,7 @@ Apache 2.0 - See [LICENSE] for more information.
117124

118125

119126
[CONTRIBUTING]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CONTRIBUTING.md
127+
[code-of-conduct]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CODE_OF_CONDUCT.md
120128
[LICENSE]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/LICENSE
121129
[TESTING]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/TESTING.md#testing-code-that-uses-storage
122130
[cloud-platform]: https://cloud.google.com/

‎branches/update-datastore/gcloud-java-storage/src/main/java/com/google/gcloud/spi/DefaultStorageRpc.java‎

Lines changed: 2 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@
3535
import com.google.api.client.googleapis.batch.json.JsonBatchCallback;
3636
import com.google.api.client.googleapis.json.GoogleJsonError;
3737
import com.google.api.client.googleapis.json.GoogleJsonResponseException;
38+
import com.google.api.client.googleapis.media.MediaHttpDownloader;
3839
import com.google.api.client.http.ByteArrayContent;
3940
import com.google.api.client.http.GenericUrl;
4041
import com.google.api.client.http.HttpHeaders;
@@ -58,18 +59,16 @@
5859
import com.google.api.services.storage.model.Objects;
5960
import com.google.api.services.storage.model.StorageObject;
6061
import com.google.common.base.MoreObjects;
61-
import com.google.common.collect.ImmutableList;
6262
import com.google.common.collect.ImmutableSet;
63-
import com.google.common.collect.Lists;
6463
import com.google.common.collect.Maps;
64+
import com.google.common.primitives.Ints;
6565
import com.google.gcloud.storage.StorageException;
6666
import com.google.gcloud.storage.StorageOptions;
6767

6868
import java.io.ByteArrayOutputStream;
6969
import java.io.IOException;
7070
import java.io.InputStream;
7171
import java.util.ArrayList;
72-
import java.util.Iterator;
7372
import java.util.List;
7473
import java.util.Map;
7574
import java.util.Set;
@@ -83,7 +82,6 @@ public class DefaultStorageRpc implements StorageRpc {
8382
// see: https://cloud.google.com/storage/docs/concepts-techniques#practices
8483
private static final Set<Integer> RETRYABLE_CODES = ImmutableSet.of(504, 503, 502, 500, 429, 408);
8584
private static final long MEGABYTE = 1024L * 1024L;
86-
private static final int MAX_BATCH_DELETES = 100;
8785

8886
public DefaultStorageRpc(StorageOptions options) {
8987
HttpTransport transport = options.httpTransportFactory().create();
@@ -363,24 +361,6 @@ public byte[] load(StorageObject from, Map<Option, ?> options)
363361

364362
@Override
365363
public BatchResponse batch(BatchRequest request) throws StorageException {
366-
List<List<Tuple<StorageObject, Map<Option, ?>>>> partitionedToDelete =
367-
Lists.partition(request.toDelete, MAX_BATCH_DELETES);
368-
Iterator<List<Tuple<StorageObject, Map<Option, ?>>>> iterator = partitionedToDelete.iterator();
369-
BatchRequest chunkRequest = new BatchRequest(iterator.hasNext() ? iterator.next() :
370-
ImmutableList.<Tuple<StorageObject, Map<Option, ?>>>of(), request.toUpdate, request.toGet);
371-
BatchResponse response = batchChunk(chunkRequest);
372-
Map<StorageObject, Tuple<Boolean, StorageException>> deletes =
373-
Maps.newHashMapWithExpectedSize(request.toDelete.size());
374-
deletes.putAll(response.deletes);
375-
while (iterator.hasNext()) {
376-
chunkRequest = new BatchRequest(iterator.next(), null, null);
377-
BatchResponse deleteBatchResponse = batchChunk(chunkRequest);
378-
deletes.putAll(deleteBatchResponse.deletes);
379-
}
380-
return new BatchResponse(deletes, response.updates, response.gets);
381-
}
382-
383-
private BatchResponse batchChunk(BatchRequest request) {
384364
com.google.api.client.googleapis.batch.BatchRequest batch = storage.batch();
385365
final Map<StorageObject, Tuple<Boolean, StorageException>> deletes =
386366
Maps.newConcurrentMap();

‎branches/update-datastore/gcloud-java-storage/src/test/java/com/google/gcloud/storage/ITStorageTest.java‎

Lines changed: 0 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@
2525
import static org.junit.Assert.assertTrue;
2626
import static org.junit.Assert.fail;
2727

28-
import com.google.api.client.util.Lists;
2928
import com.google.common.collect.ImmutableList;
3029
import com.google.common.collect.ImmutableMap;
3130
import com.google.gcloud.Page;
@@ -66,7 +65,6 @@ public class ITStorageTest {
6665
private static final String CONTENT_TYPE = "text/plain";
6766
private static final byte[] BLOB_BYTE_CONTENT = {0xD, 0xE, 0xA, 0xD};
6867
private static final String BLOB_STRING_CONTENT = "Hello Google Cloud Storage!";
69-
private static final int MAX_BATCH_DELETES = 100;
7068

7169
@BeforeClass
7270
public static void beforeClass() {
@@ -625,54 +623,6 @@ public void testBatchRequest() {
625623
assertTrue(deleteResponse.deletes().get(1).get());
626624
}
627625

628-
@Test
629-
public void testBatchRequestManyDeletes() {
630-
List<BlobId> blobsToDelete = Lists.newArrayListWithCapacity(2 * MAX_BATCH_DELETES);
631-
for (int i = 0; i < 2 * MAX_BATCH_DELETES; i++) {
632-
blobsToDelete.add(BlobId.of(BUCKET, "test-batch-request-many-deletes-blob-" + i));
633-
}
634-
BatchRequest.Builder builder = BatchRequest.builder();
635-
for (BlobId blob : blobsToDelete) {
636-
builder.delete(blob);
637-
}
638-
String sourceBlobName1 = "test-batch-request-many-deletes-source-blob-1";
639-
String sourceBlobName2 = "test-batch-request-many-deletes-source-blob-2";
640-
BlobInfo sourceBlob1 = BlobInfo.builder(BUCKET, sourceBlobName1).build();
641-
BlobInfo sourceBlob2 = BlobInfo.builder(BUCKET, sourceBlobName2).build();
642-
assertNotNull(storage.create(sourceBlob1));
643-
assertNotNull(storage.create(sourceBlob2));
644-
BlobInfo updatedBlob2 = sourceBlob2.toBuilder().contentType(CONTENT_TYPE).build();
645-
646-
BatchRequest updateRequest = builder
647-
.get(BUCKET, sourceBlobName1)
648-
.update(updatedBlob2)
649-
.build();
650-
BatchResponse response = storage.apply(updateRequest);
651-
assertEquals(2 * MAX_BATCH_DELETES, response.deletes().size());
652-
assertEquals(1, response.updates().size());
653-
assertEquals(1, response.gets().size());
654-
655-
// Check deletes
656-
for (BatchResponse.Result<Boolean> deleteResult : response.deletes()) {
657-
assertFalse(deleteResult.failed());
658-
assertFalse(deleteResult.get());
659-
}
660-
661-
// Check updates
662-
BlobInfo remoteUpdatedBlob2 = response.updates().get(0).get();
663-
assertEquals(sourceBlob2.bucket(), remoteUpdatedBlob2.bucket());
664-
assertEquals(sourceBlob2.name(), remoteUpdatedBlob2.name());
665-
assertEquals(updatedBlob2.contentType(), remoteUpdatedBlob2.contentType());
666-
667-
// Check gets
668-
BlobInfo remoteBlob1 = response.gets().get(0).get();
669-
assertEquals(sourceBlob1.bucket(), remoteBlob1.bucket());
670-
assertEquals(sourceBlob1.name(), remoteBlob1.name());
671-
672-
assertTrue(storage.delete(BUCKET, sourceBlobName1));
673-
assertTrue(storage.delete(BUCKET, sourceBlobName2));
674-
}
675-
676626
@Test
677627
public void testBatchRequestFail() {
678628
String blobName = "test-batch-request-blob-fail";

‎branches/update-datastore/gcloud-java/README.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,11 @@ If you are using SBT, add this to your dependencies
3737
libraryDependencies += "com.google.gcloud" % "gcloud-java" % "0.0.12"
3838
```
3939

40+
Troubleshooting
41+
---------------
42+
43+
To get help, follow the `gcloud-java` links in the `gcloud-*` [shared Troubleshooting document](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/troubleshooting).
44+
4045
Java Versions
4146
-------------
4247

@@ -56,7 +61,9 @@ Contributing
5661

5762
Contributions to this library are always welcome and highly encouraged.
5863

59-
See [CONTRIBUTING] for more information on how to get started.
64+
See `gcloud-java`'s [CONTRIBUTING] documentation and the `gcloud-*` [shared documentation](https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/contributing) for more information on how to get started.
65+
66+
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more information.
6067

6168
License
6269
-------
@@ -65,6 +72,7 @@ Apache 2.0 - See [LICENSE] for more information.
6572

6673

6774
[CONTRIBUTING]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CONTRIBUTING.md
75+
[code-of-conduct]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CODE_OF_CONDUCT.md
6876
[LICENSE]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/LICENSE
6977
[cloud-platform]: https://cloud.google.com/
7078
[cloud-datastore]: https://cloud.google.com/datastore/docs

0 commit comments

Comments
 (0)