Skip to content
This repository was archived by the owner on Mar 4, 2026. It is now read-only.

Commit 18be4d0

Browse files
gcf-owl-bot[bot]alexander-fenster
gcf-owl-bot[bot]
and
alexander-fenster
authored
feat: add aggregation query APIs (#1765)
* chore: Update Firestore Bazel files to refer to correct service config YAML files PiperOrigin-RevId: 472218602 Source-Link: googleapis/googleapis@1139217 Source-Link: googleapis/googleapis-gen@5fcd3a7 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNWZjZDNhNzFiMTk2Yjk2NDdjZDYxNmNjMzlhYTk4YWU3ZTBmOTY2MiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add firestore aggregation query apis to the stable googleapis branch PiperOrigin-RevId: 473753776 Source-Link: googleapis/googleapis@a8c6c7c Source-Link: googleapis/googleapis-gen@6e3b0d6 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNmUzYjBkNmVhZDQyNjVjYTZmMGFkM2UxODI5ZjRlM2E1YmMxMDlhMiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * chore: use gapic-generator-typescript v2.17.0 PiperOrigin-RevId: 474338479 Source-Link: googleapis/googleapis@d5d35e0 Source-Link: googleapis/googleapis-gen@efcd3f9 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZWZjZDNmOTM5NjJhMTAzZjY4ZjAwM2UyYTFlZWNkZTZmYTIxNmEyNyJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * fix: update protos * fix: rewrite tests to load proper proto jsons * fix: formatting * fix: add LRO protos to clients Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Alexander Fenster <[email protected]>
1 parent a1717ff commit 18be4d0

24 files changed

Lines changed: 43696 additions & 29248 deletions

‎dev/protos/firestore_admin_v1_proto_api.d.ts‎

Lines changed: 4288 additions & 3456 deletions
Large diffs are not rendered by default.

‎dev/protos/firestore_admin_v1_proto_api.js‎

Lines changed: 11837 additions & 9842 deletions
Large diffs are not rendered by default.

‎dev/protos/firestore_v1_proto_api.d.ts‎

Lines changed: 2816 additions & 1399 deletions
Large diffs are not rendered by default.

‎dev/protos/firestore_v1_proto_api.js‎

Lines changed: 7692 additions & 4423 deletions
Large diffs are not rendered by default.

‎dev/protos/firestore_v1beta1_proto_api.d.ts‎

Lines changed: 2484 additions & 1477 deletions
Large diffs are not rendered by default.

‎dev/protos/firestore_v1beta1_proto_api.js‎

Lines changed: 6724 additions & 4396 deletions
Large diffs are not rendered by default.

‎dev/protos/google/firestore/v1/aggregation_result.proto‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
// Copyright 2022 Google LLC
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
syntax = "proto3";
16+
17+
package google.firestore.v1;
18+
19+
import "google/firestore/v1/document.proto";
20+
21+
option csharp_namespace = "Google.Cloud.Firestore.V1";
22+
option go_package = "google.golang.org/genproto/googleapis/firestore/v1;firestore";
23+
option java_multiple_files = true;
24+
option java_outer_classname = "AggregationResultProto";
25+
option java_package = "com.google.firestore.v1";
26+
option objc_class_prefix = "GCFS";
27+
option php_namespace = "Google\\Cloud\\Firestore\\V1";
28+
option ruby_package = "Google::Cloud::Firestore::V1";
29+
30+
// The result of a single bucket from a Firestore aggregation query.
31+
//
32+
// The keys of `aggregate_fields` are the same for all results in an aggregation
33+
// query, unlike document queries which can have different fields present for
34+
// each result.
35+
message AggregationResult {
36+
// The result of the aggregation functions, ex: `COUNT(*) AS total_docs`.
37+
//
38+
// The key is the [alias][google.firestore.v1.StructuredAggregationQuery.Aggregation.alias]
39+
// assigned to the aggregation function on input and the size of this map
40+
// equals the number of aggregation functions in the query.
41+
map<string, Value> aggregate_fields = 2;
42+
}

‎dev/protos/google/firestore/v1/firestore.proto‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ package google.firestore.v1;
1919
import "google/api/annotations.proto";
2020
import "google/api/client.proto";
2121
import "google/api/field_behavior.proto";
22+
import "google/firestore/v1/aggregation_result.proto";
2223
import "google/firestore/v1/common.proto";
2324
import "google/firestore/v1/document.proto";
2425
import "google/firestore/v1/query.proto";
@@ -135,6 +136,29 @@ service Firestore {
135136
};
136137
}
137138

139+
// Runs an aggregation query.
140+
//
141+
// Rather than producing [Document][google.firestore.v1.Document] results like [Firestore.RunQuery][google.firestore.v1.Firestore.RunQuery],
142+
// this API allows running an aggregation to produce a series of
143+
// [AggregationResult][google.firestore.v1.AggregationResult] server-side.
144+
//
145+
// High-Level Example:
146+
//
147+
// ```
148+
// -- Return the number of documents in table given a filter.
149+
// SELECT COUNT(*) FROM ( SELECT * FROM k where a = true );
150+
// ```
151+
rpc RunAggregationQuery(RunAggregationQueryRequest) returns (stream RunAggregationQueryResponse) {
152+
option (google.api.http) = {
153+
post: "/v1/{parent=projects/*/databases/*/documents}:runAggregationQuery"
154+
body: "*"
155+
additional_bindings {
156+
post: "/v1/{parent=projects/*/databases/*/documents/*/**}:runAggregationQuery"
157+
body: "*"
158+
}
159+
};
160+
}
161+
138162
// Partitions a query by returning partition cursors that can be used to run
139163
// the query in parallel. The returned partition cursors are split points that
140164
// can be used by RunQuery as starting/end points for the query results.
@@ -534,6 +558,61 @@ message RunQueryResponse {
534558
}
535559
}
536560

561+
// The request for [Firestore.RunAggregationQuery][google.firestore.v1.Firestore.RunAggregationQuery].
562+
message RunAggregationQueryRequest {
563+
// Required. The parent resource name. In the format:
564+
// `projects/{project_id}/databases/{database_id}/documents` or
565+
// `projects/{project_id}/databases/{database_id}/documents/{document_path}`.
566+
// For example:
567+
// `projects/my-project/databases/my-database/documents` or
568+
// `projects/my-project/databases/my-database/documents/chatrooms/my-chatroom`
569+
string parent = 1 [(google.api.field_behavior) = REQUIRED];
570+
571+
// The query to run.
572+
oneof query_type {
573+
// An aggregation query.
574+
StructuredAggregationQuery structured_aggregation_query = 2;
575+
}
576+
577+
// The consistency mode for the query, defaults to strong consistency.
578+
oneof consistency_selector {
579+
// Run the aggregation within an already active transaction.
580+
//
581+
// The value here is the opaque transaction ID to execute the query in.
582+
bytes transaction = 4;
583+
584+
// Starts a new transaction as part of the query, defaulting to read-only.
585+
//
586+
// The new transaction ID will be returned as the first response in the
587+
// stream.
588+
TransactionOptions new_transaction = 5;
589+
590+
// Executes the query at the given timestamp.
591+
//
592+
// Requires:
593+
//
594+
// * Cannot be more than 270 seconds in the past.
595+
google.protobuf.Timestamp read_time = 6;
596+
}
597+
}
598+
599+
// The response for [Firestore.RunAggregationQuery][google.firestore.v1.Firestore.RunAggregationQuery].
600+
message RunAggregationQueryResponse {
601+
// A single aggregation result.
602+
//
603+
// Not present when reporting partial progress.
604+
AggregationResult result = 1;
605+
606+
// The transaction that was started as part of this request.
607+
//
608+
// Only present on the first response when the request requested to start
609+
// a new transaction.
610+
bytes transaction = 2;
611+
612+
// The time at which the aggregate value is valid for.
613+
google.protobuf.Timestamp read_time = 3;
614+
}
615+
537616
// The request for [Firestore.PartitionQuery][google.firestore.v1.Firestore.PartitionQuery].
538617
message PartitionQueryRequest {
539618
// Required. The parent resource name. In the format:

‎dev/protos/google/firestore/v1/query.proto‎

Lines changed: 141 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -228,8 +228,13 @@ message StructuredQuery {
228228
DESCENDING = 2;
229229
}
230230

231-
// A reference to a field, such as `max(messages.time) as max_time`.
231+
// A reference to a field in a document, ex: `stats.operations`.
232232
message FieldReference {
233+
// The relative path of the document being referenced.
234+
//
235+
// Requires:
236+
//
237+
// * Conform to [document field name][google.firestore.v1.Document.fields] limitations.
233238
string field_path = 2;
234239
}
235240

@@ -273,25 +278,154 @@ message StructuredQuery {
273278
// `WHERE __name__ > ... AND a > 1 ORDER BY a ASC, __name__ ASC`
274279
repeated Order order_by = 4;
275280

276-
// A starting point for the query results.
281+
// A potential prefix of a position in the result set to start the query at.
282+
//
283+
// The ordering of the result set is based on the `ORDER BY` clause of the
284+
// original query.
285+
//
286+
// ```
287+
// SELECT * FROM k WHERE a = 1 AND b > 2 ORDER BY b ASC, __name__ ASC;
288+
// ```
289+
//
290+
// This query's results are ordered by `(b ASC, __name__ ASC)`.
291+
//
292+
// Cursors can reference either the full ordering or a prefix of the location,
293+
// though it cannot reference more fields than what are in the provided
294+
// `ORDER BY`.
295+
//
296+
// Continuing off the example above, attaching the following start cursors
297+
// will have varying impact:
298+
//
299+
// - `START BEFORE (2, /k/123)`: start the query right before `a = 1 AND
300+
// b > 2 AND __name__ > /k/123`.
301+
// - `START AFTER (10)`: start the query right after `a = 1 AND b > 10`.
302+
//
303+
// Unlike `OFFSET` which requires scanning over the first N results to skip,
304+
// a start cursor allows the query to begin at a logical position. This
305+
// position is not required to match an actual result, it will scan forward
306+
// from this position to find the next document.
307+
//
308+
// Requires:
309+
//
310+
// * The number of values cannot be greater than the number of fields
311+
// specified in the `ORDER BY` clause.
277312
Cursor start_at = 7;
278313

279-
// A end point for the query results.
314+
// A potential prefix of a position in the result set to end the query at.
315+
//
316+
// This is similar to `START_AT` but with it controlling the end position
317+
// rather than the start position.
318+
//
319+
// Requires:
320+
//
321+
// * The number of values cannot be greater than the number of fields
322+
// specified in the `ORDER BY` clause.
280323
Cursor end_at = 8;
281324

282-
// The number of results to skip.
325+
// The number of documents to skip before returning the first result.
283326
//
284-
// Applies before limit, but after all other constraints. Must be >= 0 if
285-
// specified.
327+
// This applies after the constraints specified by the `WHERE`, `START AT`, &
328+
// `END AT` but before the `LIMIT` clause.
329+
//
330+
// Requires:
331+
//
332+
// * The value must be greater than or equal to zero if specified.
286333
int32 offset = 6;
287334

288335
// The maximum number of results to return.
289336
//
290337
// Applies after all other constraints.
291-
// Must be >= 0 if specified.
338+
//
339+
// Requires:
340+
//
341+
// * The value must be greater than or equal to zero if specified.
292342
google.protobuf.Int32Value limit = 5;
293343
}
294344

345+
// Firestore query for running an aggregation over a [StructuredQuery][google.firestore.v1.StructuredQuery].
346+
message StructuredAggregationQuery {
347+
// Defines a aggregation that produces a single result.
348+
message Aggregation {
349+
// Count of documents that match the query.
350+
//
351+
// The `COUNT(*)` aggregation function operates on the entire document
352+
// so it does not require a field reference.
353+
message Count {
354+
// Optional. Optional constraint on the maximum number of documents to count.
355+
//
356+
// This provides a way to set an upper bound on the number of documents
357+
// to scan, limiting latency and cost.
358+
//
359+
// Unspecified is interpreted as no bound.
360+
//
361+
// High-Level Example:
362+
//
363+
// ```
364+
// AGGREGATE COUNT_UP_TO(1000) OVER ( SELECT * FROM k );
365+
// ```
366+
//
367+
// Requires:
368+
//
369+
// * Must be greater than zero when present.
370+
google.protobuf.Int64Value up_to = 1 [(google.api.field_behavior) = OPTIONAL];
371+
}
372+
373+
// The type of aggregation to perform, required.
374+
oneof operator {
375+
// Count aggregator.
376+
Count count = 1;
377+
}
378+
379+
// Optional. Optional name of the field to store the result of the aggregation into.
380+
//
381+
// If not provided, Firestore will pick a default name following the format
382+
// `field_<incremental_id++>`. For example:
383+
//
384+
// ```
385+
// AGGREGATE
386+
// COUNT_UP_TO(1) AS count_up_to_1,
387+
// COUNT_UP_TO(2),
388+
// COUNT_UP_TO(3) AS count_up_to_3,
389+
// COUNT_UP_TO(4)
390+
// OVER (
391+
// ...
392+
// );
393+
// ```
394+
//
395+
// becomes:
396+
//
397+
// ```
398+
// AGGREGATE
399+
// COUNT_UP_TO(1) AS count_up_to_1,
400+
// COUNT_UP_TO(2) AS field_1,
401+
// COUNT_UP_TO(3) AS count_up_to_3,
402+
// COUNT_UP_TO(4) AS field_2
403+
// OVER (
404+
// ...
405+
// );
406+
// ```
407+
//
408+
// Requires:
409+
//
410+
// * Must be unique across all aggregation aliases.
411+
// * Conform to [document field name][google.firestore.v1.Document.fields] limitations.
412+
string alias = 7 [(google.api.field_behavior) = OPTIONAL];
413+
}
414+
415+
// The base query to aggregate over.
416+
oneof query_type {
417+
// Nested structured query.
418+
StructuredQuery structured_query = 1;
419+
}
420+
421+
// Optional. Series of aggregations to apply over the results of the `structured_query`.
422+
//
423+
// Requires:
424+
//
425+
// * A minimum of one and maximum of five aggregations per query.
426+
repeated Aggregation aggregations = 3 [(google.api.field_behavior) = OPTIONAL];
427+
}
428+
295429
// A position in a query result set.
296430
message Cursor {
297431
// The values that represent a position, in the order they appear in

0 commit comments

Comments
 (0)