| title | Query Batching |
|---|---|
| subtitle | Receive query batches with the GraphOS Router |
| description | Handle multiple GraphQL requests with GraphOS Router's query batching capabilities. Aggregate operations into single HTTP requests to preserve data consistency. |
Learn how query batching can help preserve data consistency of responses, and learn how to configure the GraphOS Router to receive query batches.
Modern applications often require several requests to render a single page. This is usually the result of a component-based architecture where individual micro-frontends (MFE) make requests separately to fetch data relevant to them.
Query batching makes response data consistent for MFEs making multiple requests per page. It primarily preserves data consistency rather than improve performance.
When the underlying data is changing rapidly, the separate requests of an application may result in responses with inconsistent data. Given two requests for a single page, the values returned in their responses may be different because the data's been updated or subgraph servers briefly had different values.
To prevent this inconsistency from happening, clients can bundle multiple requests together into a batch so routers or servers can produce responses with consistent data. MFE-based UIs usually batch multiple client operations, issued close together, into a single HTTP request. Both Apollo Client and Apollo Server support this.
The router's batching support is provided by two sets of functionality:
- client batching
- subgraph batching
With client batching, the router accepts batched requests from a client and processes each request of a batch separately. Consequently, the router doesn't present requests to subgraphs in batch form, so subgraphs must process the requests of a batch individually.
With subgraph batching, the router analyzes input client batch requests and issues batch requests to subgraphs. Subgraph batching is an extension to client batching and requires participating subgraphs to support batching requests. See the examples below to see illustrations of how this works in practice.
The GraphOS Router supports client and subgraph query batching.
The GraphOS Router must be configured to receive query batches, otherwise it rejects them. When processing a batch, the router deserializes and processes each operation of a batch independently. It responds to the client only after all operations of the batch have been completed. Each operation executes concurrently with respect to other operations in the batch.
If you’re using Apollo Client, you can leverage the built-in support for batching to reduce the number of individual operations sent to the router.
Once configured, Apollo Client automatically combines multiple operations into a single HTTP request. The number of operations within a batch is client-configurable, including the maximum number in a batch and the maximum duration to wait for operations to accumulate before sending the batch.
Both the GraphOS Router and client need to be configured to support query batching.
By default, receiving client query batches is not enabled in the GraphOS Router.
To enable query batching, set the following fields in your router.yaml configuration file:
batching:
enabled: true
mode: batch_http_link| Attribute | Description | Valid Values | Default Value |
|---|---|---|---|
enabled |
Flag to enable reception of client query batches | boolean | false |
mode |
Supported client batching mode | batch_http_link: the client uses Apollo Link and its BatchHttpLink link. |
No Default |
maximum_size |
Maximum number of queries in a client batch (optional) | integer | null (no limit on number of queries) |
If client query batching is enabled, and the router's subgraphs support query batching, then subgraph query batching can be enabled by setting the following fields in your router.yaml configuration file:
batching:
enabled: true
mode: batch_http_link
subgraph:
# Enable batching on all subgraphs
all:
enabled: truebatching:
enabled: true
mode: batch_http_link
subgraph:
# Disable batching on all subgraphs
all:
enabled: false
# Configure(over-ride) batching support per subgraph
subgraphs:
subgraph_1:
enabled: true
subgraph_2:
enabled: true-
The router can be configured to support batching for either all subgraphs or individually enabled per subgraph.
-
There are limitations on the ability of the router to preserve batches from the client request into the subgraph requests. In particular, certain forms of queries will require data to be present before they are processed. Consequently, the router will only be able to generate batches from queries which are processed which don't contain such constraints. This may result in the router issuing multiple batches or requests.
-
If query deduplication or entity caching are enabled, they will not apply to batched queries. Batching will take precedence over query deduplication and entity caching. Query deduplication and entity caching will still be performed for non-batched queries.
This example shows how the router can batch subgraph requests in the most efficient scenario, where the queries of a batch don't have required fetch constraints.
Assume the federated graph contains three subgraphs: accounts, products, and reviews.
The input client query to the federated graph:
[
{"query":"query MeQuery1 {\n me {\n id\n }\n}"}
{"query":"query MeQuery2 {\n me {\n name\n }\n}"},
{"query":"query MeQuery3 {\n me {\n id\n }\n}"}
{"query":"query MeQuery4 {\n me {\n name\n }\n}"},
{"query":"query MeQuery5 {\n me {\n id\n }\n}"}
{"query":"query MeQuery6 {\n me {\n name\n }\n}"},
{"query":"query MeQuery7 {\n me {\n id\n }\n}"}
{"query":"query MeQuery8 {\n me {\n name\n }\n}"},
{"query":"query MeQuery9 {\n me {\n id\n }\n}"}
{"query":"query MeQuery10 {\n me {\n name\n }\n}"},
{"query":"query MeQuery11 {\n me {\n id\n }\n}"}
{"query":"query MeQuery12 {\n me {\n name\n }\n}"},
{"query":"query MeQuery13 {\n me {\n id\n }\n}"}
{"query":"query MeQuery14 {\n me {\n name\n }\n}"},
{"query":"query MeQuery15 {\n me {\n id\n }\n}"}
]From the input query, the router generates a set of subgraph queries:
"query MeQuery1__accounts__0{me{id}}",
"query MeQuery2__accounts__0{me{name}}",
"query MeQuery3__accounts__0{me{id}}",
"query MeQuery4__accounts__0{me{name}}",
"query MeQuery5__accounts__0{me{id}}",
"query MeQuery6__accounts__0{me{name}}",
"query MeQuery7__accounts__0{me{id}}",
"query MeQuery8__accounts__0{me{name}}",
"query MeQuery9__accounts__0{me{id}}",
"query MeQuery10__accounts__0{me{name}}",
"query MeQuery11__accounts__0{me{id}}",
"query MeQuery12__accounts__0{me{name}}",
"query MeQuery13__accounts__0{me{id}}",
"query MeQuery14__accounts__0{me{name}}",
"query MeQuery15__accounts__0{me{id}}",
All of the queries can be combined into a single batch. So instead of 15 (non-batch) subgraph fetches, the router only has to make one fetch.
| Subgraph | Fetch Count (without) | Fetch Count (with) |
|---|---|---|
| accounts | 15 | 1 |
This example shows how the router might batch subgraph requests for a graph, where the client batch contains a query for an entity.
Assume the federated graph contains three subgraphs: accounts, products, and reviews.
The input client query to the federated graph:
[
{"query":"query MeQuery1 {\n me {\n id\n }\n}"},
{"query":"query MeQuery2 {\n me {\n reviews {\n body\n }\n }\n}"},
{"query":"query MeQuery3 {\n topProducts {\n upc\n reviews {\n author {\n name\n }\n }\n }\n me {\n name\n }\n}"},
{"query":"query MeQuery4 {\n me {\n name\n }\n}"},
{"query":"query MeQuery5 {\n me {\n id\n }\n}"}
]From the input query, the router generates a set of subgraph queries:
"query MeQuery1__accounts__0{me{id}}",
"query MeQuery2__accounts__0{me{__typename id}}",
"query MeQuery3__products__0{topProducts{__typename upc}}",
"query MeQuery3__accounts__3{me{name}}",
"query MeQuery4__accounts__0{me{name}}",
"query MeQuery5__accounts__0{me{id}}",
"query MeQuery2__reviews__1($representations:[_Any!]!){_entities(representations:$representations){...on User{reviews{body}}}}",
"query MeQuery3__reviews__1($representations:[_Any!]!){_entities(representations:$representations){...on Product{reviews{author{__typename id}}}}}",
"query MeQuery3__accounts__2($representations:[_Any!]!){_entities(representations:$representations){...on User{name}}}",
The first six queries can be combined into two batches—one for accounts and one for products. They must be fetched before the final three queries can be executed individually.
Overall, without subgraph batching, the router would make nine fetches in total across the three subgraphs, but with subgraph batching, that total is reduced to five fetches.
| Subgraph | Fetch Count (without) | Fetch Count (with) |
|---|---|---|
| accounts | 6 | 2 |
| products | 1 | 1 |
| reviews | 2 | 2 |
To enable batching in an Apollo client, configure BatchHttpLink. For details on implementing BatchHttpLink, see batching operations.
If the router receives a query batch from a client, and batching is not enabled, the router sends a BATCHING_NOT_ENABLED error to the client.
Metrics in the GraphOS Router for query batching:
| Name | Attributes | Description |
|---|---|---|
|
mode [subgraph] |
Counter for the number of received (from client) or dispatched (to subgraph) batches. |
|
|
mode [subgraph] |
Histogram for the size of received batches. |
The subgraph attribute is optional. If the attribute isn't present, the metric identifies batches received from clients. If the attribute is present, the metric identifies batches sent to a particular subgraph.
A query batch is an array of operations.
[
query MyFirstQuery {
me {
id
}
},
query MySecondQuery {
me {
name
}
}
]Responses are provided in JSON array, with the order of responses matching the order of operations in the query batch.
[
{"data":{"me":{"id":"1"}}},
{"data":{"me":{"name":"Ada Lovelace"}}}
]If a batch of queries cannot be processed, the entire batch fails.
For example, this batch request is invalid because it has two commas to separate the constituent queries:
[
query MyFirstQuery {
me {
id
}
},,
query MySecondQuery {
me {
name
}
}
]As a result, the router returns an invalid batch error:
{"errors":
[
{"message":"Invalid GraphQL request","extensions":{"details":"failed to deserialize the request body into JSON: expected value at line 1 column 54","code":"INVALID_GRAPHQL_REQUEST"}}
]
}If the number of queries provided exceeds the maximum batch size, the entire batch fails.
For example, this configuration sets a batch size limit of 2, but three queries are provided:
batching:
enabled: true
mode: batch_http_link
maximum_size: 2[
query MyFirstQuery {
me {
id
}
},
query MySecondQuery {
me {
name
}
},
query MyThirdQuery {
me {
name
}
}
]As a result, the router returns an error:
{
"errors": [
{
"message": "Invalid GraphQL request",
"extensions": {
"details": "Batch limits exceeded: you provided a batch with 3 entries, but the configured maximum router batch size is 2",
"code": "BATCH_LIMIT_EXCEEDED"
}
}
]
}If a single query in a batch cannot be processed, this results in an individual error.
For example, the query MyFirstQuery is accessing a field that doesn't exist, while the rest of the batch query is valid.
[
query MyFirstQuery {
me {
thisfielddoesnotexist
}
},
query MySecondQuery {
me {
name
}
}
]As a result, an error is returned for the individual invalid query and the other (valid) query returns a response.
[
{"errors":
[
{"message":"cannot query field 'thisfielddoesnotexist' on type 'User'",
"extensions":{"type":"User","field":"thisfielddoesnotexist","code":"INVALID_FIELD"}
}
]
},
{"data":{"me":{"name":"Ada Lovelace"}}}
]When batching is enabled, any batch operation that results in a stream of responses is unsupported, including: