Skip to content

Commit 4d0381d

Browse files
tswasttswast
authored
---
yaml --- r: 8241 b: refs/heads/master c: 83d85fc h: refs/heads/master i: 8239: c6d7983
1 parent 8694bd7 commit 4d0381d

104 files changed

Lines changed: 1466 additions & 284 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎[refs]‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
refs/heads/master: edfb43a8a10f2b3c4f97e7cd2581f45c2ca442aa
2+
refs/heads/master: 83d85fc21ed8718768e0a93e7de7a443e4ae4d30
33
refs/heads/travis: 47e4fee4fd5af9b2a8ce46f23c72ec95f9b195b2
44
refs/heads/gh-pages: 0114589947fea527ea3831330789accedbecf45c
55
refs/tags/0.0.9: 22f1839238f66c39e67ed4dfdcd273b1ae2e8444

‎trunk/.travis.yml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,6 @@ before_install:
1616
- cp target/travis/logging.properties logging.properties
1717
install: mvn install --quiet -Djava.util.logging.config.file=logging.properties -DskipTests=true
1818
-Dgpg.skip=true
19-
cache:
20-
directories:
21-
- "$HOME/google-cloud-sdk/"
2219
script:
2320
- travis_wait 60 utilities/verify.sh
2421
after_success:

‎trunk/APPENGINE.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
Use Google Cloud Java Client with Google App Engine
22
=================================================
33

4+
** Note: the following instructions are not required for App Engine Flex and Java 8 standard environment. **
5+
46
Quickstart
57
----------
68
This file provides extra instructions to set up Google Cloud Java Client with Google App Engine.
@@ -30,4 +32,4 @@ libraryDependencies += "com.google.appengine" % "appengine-api-1.0-sdk" % "1.9.5
3032
To test with local dev server, run
3133
```
3234
mvn appengine:devserver
33-
```
35+
```

‎trunk/README.md‎

Lines changed: 68 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -54,20 +54,54 @@ If you are using Maven, add this to your pom.xml file
5454
<dependency>
5555
<groupId>com.google.cloud</groupId>
5656
<artifactId>google-cloud</artifactId>
57-
<version>0.20.3-alpha</version>
57+
<version>0.21.1-alpha</version>
5858
</dependency>
5959
```
6060
If you are using Gradle, add this to your dependencies
6161
```Groovy
62-
compile 'com.google.cloud:google-cloud:0.20.3-alpha'
62+
compile 'com.google.cloud:google-cloud:0.21.1-alpha'
6363
```
6464
If you are using SBT, add this to your dependencies
6565
```Scala
66-
libraryDependencies += "com.google.cloud" % "google-cloud" % "0.20.3-alpha"
66+
libraryDependencies += "com.google.cloud" % "google-cloud" % "0.21.1-alpha"
6767
```
6868

6969
For running on Google App Engine, see [more instructions here](./APPENGINE.md).
7070

71+
Resolving dependency conflicts
72+
----------
73+
Optionally, if you encounter dependency conflicts, you may specify google-cloud-pom as a ["bill of materials"](https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Importing_Dependencies) to ensure that internal dependencies of google-cloud and your project are in sync.
74+
75+
If you are using Maven, add this to your pom.xml file
76+
```xml
77+
<dependencyManagement>
78+
<dependencies>
79+
<dependency>
80+
<groupId>com.google.cloud</groupId>
81+
<artifactId>google-cloud-pom</artifactId>
82+
<version>0.21.1-alpha</version>
83+
<type>pom</type>
84+
<scope>import</scope>
85+
</dependency>
86+
</dependencies>
87+
</dependencyManagement>
88+
```
89+
If you are using Gradle, add the following plugin at the beginning of your build.gradle file
90+
91+
```Groovy
92+
plugins {
93+
id "io.spring.dependency-management" version "1.0.3.RELEASE"
94+
}
95+
```
96+
Then add the following in your build.gradle file
97+
```Groovy
98+
dependencyManagement {
99+
imports {
100+
mavenBom 'com.google.cloud:google-cloud-pom:0.21.1-alpha'
101+
}
102+
}
103+
```
104+
71105
Example Applications
72106
--------------------
73107

@@ -126,6 +160,13 @@ Most `google-cloud` libraries require a project ID. There are multiple ways to
126160
5. The Google Cloud SDK project ID
127161
6. The Compute Engine project ID
128162

163+
In cases where the library may expect a project ID explicitly, we provide a helper that can provide the inferred project ID:
164+
```java
165+
import com.google.cloud.ServiceOptions;
166+
...
167+
String projectId = ServiceOptions.getDefaultProjectId();
168+
```
169+
129170
Authentication
130171
--------------
131172

@@ -876,12 +917,31 @@ Java 7 or above is required for using the clients in this repository.
876917
Supported Platforms
877918
-------------------
878919
879-
This client is supported on Mac OS X, Windows and Linux (excluding Android and Alpine).
880-
Google Cloud Platform environments currently supported include GCE, GKE and GAE Flex.
881-
GAE Standard is not currently supported.
920+
Clients in this repository use either HTTP or gRPC for the transport layer. All
921+
HTTP-based clients should work in all environments.
922+
923+
For clients that use gRPC, the supported platforms are constrained by the platforms
924+
that [Forked Tomcat Native](http://netty.io/wiki/forked-tomcat-native.html) supports,
925+
which for architectures means only x86_64, and for operating systems means Mac OS X,
926+
Windows, and Linux. Additionally, gRPC constrains the use of platforms with
927+
threading restrictions.
928+
929+
Thus, the following are not supported:
930+
931+
- Android
932+
- Alpine Linux (due to netty-tcnative requiring glibc, which is not present on Alpine)
933+
- Raspberry Pi (since it runs on the ARM architecture)
934+
- Google App Engine Standard Java 7
935+
936+
The following environments should work (among others):
882937
883-
Spring Boot users : Native Tomcat is not currently supported. Please use [embedded Jetty](https://docs.spring.io/spring-boot/docs/current/reference/html/howto-embedded-servlet-containers.html#howto-use-jetty-instead-of-tomcat)
884-
to get your application working with this client.
938+
- standalone Windows on x86_64
939+
- standalone Mac OS X on x86_64
940+
- standalone Linux on x86_64
941+
- Google Compute Engine (GCE)
942+
- Google Container Engine (GKE)
943+
- Google App Engine Standard Java 8 (GAE Std J8)
944+
- Google App Engine Flex (GAE Flex)
885945
886946
Testing
887947
-------

‎trunk/google-cloud-bigquery/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,16 +22,16 @@ If you are using Maven, add this to your pom.xml file
2222
<dependency>
2323
<groupId>com.google.cloud</groupId>
2424
<artifactId>google-cloud-bigquery</artifactId>
25-
<version>0.20.3-beta</version>
25+
<version>0.21.1-beta</version>
2626
</dependency>
2727
```
2828
If you are using Gradle, add this to your dependencies
2929
```Groovy
30-
compile 'com.google.cloud:google-cloud-bigquery:0.20.3-beta'
30+
compile 'com.google.cloud:google-cloud-bigquery:0.21.1-beta'
3131
```
3232
If you are using SBT, add this to your dependencies
3333
```Scala
34-
libraryDependencies += "com.google.cloud" % "google-cloud-bigquery" % "0.20.3-beta"
34+
libraryDependencies += "com.google.cloud" % "google-cloud-bigquery" % "0.21.1-beta"
3535
```
3636

3737
Example Application

‎trunk/google-cloud-bigquery/pom.xml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
33
<modelVersion>4.0.0</modelVersion>
44
<artifactId>google-cloud-bigquery</artifactId>
5-
<version>0.20.4-beta-SNAPSHOT</version>
5+
<version>0.21.2-beta-SNAPSHOT</version>
66
<packaging>jar</packaging>
77
<name>Google Cloud BigQuery</name>
88
<url>https://github.com/GoogleCloudPlatform/google-cloud-java/tree/master/google-cloud-bigquery</url>
@@ -12,7 +12,7 @@
1212
<parent>
1313
<groupId>com.google.cloud</groupId>
1414
<artifactId>google-cloud-pom</artifactId>
15-
<version>0.20.4-alpha-SNAPSHOT</version>
15+
<version>0.21.2-alpha-SNAPSHOT</version>
1616
</parent>
1717
<properties>
1818
<site.installationModule>google-cloud-bigquery</site.installationModule>

‎trunk/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQuery.java‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -935,7 +935,7 @@ Page<List<FieldValue>> listTableData(String datasetId, String tableId,
935935
boolean cancel(JobId jobId);
936936

937937
/**
938-
* Runs the query associated with the request.
938+
* Runs the query associated with the request, using an internally-generated random JobId.
939939
*
940940
* <p>Example of running a query.
941941
* <pre> {@code
@@ -986,6 +986,19 @@ Page<List<FieldValue>> listTableData(String datasetId, String tableId,
986986
*/
987987
QueryResponse query(QueryRequest request);
988988

989+
/**
990+
* Runs the query associated with the request, using the given job id.
991+
*
992+
* <p>See {@link #query(QueryRequest)} for examples on populating a {@link QueryRequest}. The
993+
* recommended way to populate a JobId is the following:
994+
*
995+
* <p>
996+
* <pre> {@code
997+
* JobId jobId = JobId.of(UUID.randomUUID().toString());
998+
* }</pre>
999+
*/
1000+
QueryResponse query(QueryRequest request, JobId jobId);
1001+
9891002
/**
9901003
* Returns results of the query associated with the provided job.
9911004
*

‎trunk/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java‎

Lines changed: 39 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,11 @@
3737
import com.google.common.collect.Iterables;
3838
import com.google.common.collect.Lists;
3939
import com.google.common.collect.Maps;
40+
import java.util.ArrayList;
41+
import java.util.Collections;
4042
import java.util.List;
4143
import java.util.Map;
44+
import java.util.UUID;
4245
import java.util.concurrent.Callable;
4346

4447
final class BigQueryImpl extends BaseService<BigQueryOptions> implements BigQuery {
@@ -560,38 +563,47 @@ public Boolean call() {
560563

561564
@Override
562565
public QueryResponse query(final QueryRequest request) {
566+
JobId jobId = JobId.of(UUID.randomUUID().toString());
567+
return query(request, jobId);
568+
}
569+
570+
@Override
571+
public QueryResponse query(final QueryRequest request, final JobId jobId) {
572+
final JobId completeJobId = jobId.setProjectId(getOptions().getProjectId());
563573
try {
564-
com.google.api.services.bigquery.model.QueryResponse results =
565-
runWithRetries(new Callable<com.google.api.services.bigquery.model.QueryResponse>() {
574+
QueryJobConfiguration queryJobConfiguration =
575+
QueryJobConfiguration.newBuilder(request.getQuery())
576+
.setPositionalParameters(request.getPositionalParameters())
577+
.setNamedParameters(request.getNamedParameters())
578+
.setDefaultDataset(request.getDefaultDataset())
579+
.setDryRun(request.dryRun())
580+
.setUseQueryCache(request.useQueryCache())
581+
.setUseLegacySql(request.useLegacySql())
582+
.build();
583+
584+
final JobInfo jobInfo =
585+
JobInfo.newBuilder(queryJobConfiguration).setJobId(completeJobId).build();
586+
587+
com.google.api.services.bigquery.model.Job jobPb =
588+
runWithRetries(new Callable<com.google.api.services.bigquery.model.Job>() {
566589
@Override
567-
public com.google.api.services.bigquery.model.QueryResponse call() {
568-
return bigQueryRpc.query(request.setProjectId(getOptions().getProjectId()).toPb());
590+
public com.google.api.services.bigquery.model.Job call() {
591+
return bigQueryRpc.create(jobInfo.toPb(),
592+
Collections.<BigQueryRpc.Option, Object>emptyMap());
569593
}
570594
}, getOptions().getRetrySettings(), EXCEPTION_HANDLER, getOptions().getClock());
571-
QueryResponse.Builder builder = QueryResponse.newBuilder();
572-
JobId completeJobId = JobId.fromPb(results.getJobReference());
573-
builder.setJobId(completeJobId);
574-
builder.setJobCompleted(results.getJobComplete());
575-
List<TableRow> rowsPb = results.getRows();
576-
if (results.getJobComplete()) {
577-
builder.setJobCompleted(true);
578-
QueryResult.Builder resultBuilder = transformQueryResults(completeJobId, rowsPb,
579-
results.getPageToken(), getOptions(), ImmutableMap.<BigQueryRpc.Option, Object>of());
580-
resultBuilder.setTotalBytesProcessed(results.getTotalBytesProcessed());
581-
resultBuilder.setCacheHit(results.getCacheHit());
582-
if (results.getSchema() != null) {
583-
resultBuilder.setSchema(Schema.fromPb(results.getSchema()));
584-
}
585-
if (results.getTotalRows() != null) {
586-
resultBuilder.setTotalRows(results.getTotalRows().longValue());
587-
}
588-
builder.setResult(resultBuilder.build());
595+
Job job = Job.fromPb(this, jobPb);
596+
597+
List<QueryResultsOption> options = new ArrayList<>();
598+
if (request.getMaxWaitTime() != null) {
599+
options.add(QueryResultsOption.maxWaitTime(request.getMaxWaitTime()));
589600
}
590-
if (results.getErrors() != null) {
591-
builder.setExecutionErrors(
592-
Lists.transform(results.getErrors(), BigQueryError.FROM_PB_FUNCTION));
601+
if (request.getPageSize() != null) {
602+
options.add(QueryResultsOption.pageSize(request.getPageSize()));
593603
}
594-
return builder.build();
604+
return getQueryResults(job.getJobId(),
605+
options.toArray(new QueryResultsOption[options.size()]));
606+
595607
} catch (RetryHelper.RetryHelperException e) {
596608
throw BigQueryException.translateAndThrow(e);
597609
}
@@ -617,6 +629,7 @@ public GetQueryResultsResponse call() {
617629
}, serviceOptions.getRetrySettings(), EXCEPTION_HANDLER, serviceOptions.getClock());
618630
QueryResponse.Builder builder = QueryResponse.newBuilder();
619631
builder.setJobId(JobId.fromPb(results.getJobReference()));
632+
builder.setNumDmlAffectedRows(results.getNumDmlAffectedRows());
620633
builder.setEtag(results.getEtag());
621634
builder.setJobCompleted(results.getJobComplete());
622635
List<TableRow> rowsPb = results.getRows();

0 commit comments

Comments
 (0)