You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4b02e22
Browse filesBrowse the repository at this point in the historyBrowse files
feat: Batch Postgres online reads across feature views
The generic OnlineStore._read_features_per_fv loop issues one online_read per
feature view, so a request spanning N feature views costs N round trips. Redis
already overrides the seam to batch; Postgres now does too.
Each feature view has its own table, so the per-view reads are combined with
UNION ALL and split apart again by a constant tag column. The tag is what keeps
them separable: the table name is not in the result set, and two feature views
can legitimately return the same entity key AND the same feature name.
Batching only pays where round trips dominate. Measured against Postgres 16 with
a 2ms round trip, a 10-view request is ~5x faster at 1 entity and ~3x at 10, but
a wash at 5000 entities, where holding every view's rows at once costs tens of MB
per in-flight request. Requests above MAX_BATCHED_READ_KEYS entity keys therefore
fall back to the generic per-view path, which processes and releases one view at
a time. The threshold is checked from the request shape before any keys are
serialized, so choosing not to batch costs nothing.
Both the sync and async paths are overridden. The generic async path already runs
the per-view queries concurrently via asyncio.gather, so the async win is N
concurrent queries becoming one rather than a change from serial. Note that
Postgres does not advertise async_supported today, so the async path is not yet
reachable from the feature server.
Also corrects four doc references that described Redis and Postgres as overriding
get_online_features; both override _read_features_per_fv.
Refs feast-dev#3259
Signed-off-by: DKilkenny <[email protected]>
Copy file name to clipboardExpand all lines: docs/how-to-guides/online-server-performance-tuning.md
+8-6Lines changed: 8 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,7 +32,9 @@ When the server processes a `get_online_features()` call, it groups the requeste
32
32
**Guideline:** For features that share the same entity key and are frequently requested together, consolidate them into a **single feature view**. This reduces the number of store round-trips per request. Split feature views only when features have different entities, different materialization schedules, or different data source update frequencies.
33
33
34
34
{% hint style="info" %}
35
-
**Redis exception:** The Redis online store overrides `get_online_features()` to batch all `HMGET` commands across every feature view into a **single pipeline execution**. Because all feature views for the same entity share one Redis hash key, the number of Redis round trips is always **1**, regardless of how many feature views the request touches. This means the "fewer feature views" guideline is less critical for Redis than for other stores — but consolidating feature views still reduces serialization and protobuf overhead at the application layer.
35
+
**Redis exception:** The Redis online store overrides `_read_features_per_fv()` to batch all `HMGET` commands across every feature view into a **single pipeline execution**. Because all feature views for the same entity share one Redis hash key, the number of Redis round trips is always **1**, regardless of how many feature views the request touches. This means the "fewer feature views" guideline is less critical for Redis than for other stores — but consolidating feature views still reduces serialization and protobuf overhead at the application layer.
36
+
37
+
**PostgreSQL exception:** PostgreSQL stores each feature view in its own table and overrides `_read_features_per_fv()` to combine the per-view reads into a **single `UNION ALL` statement**, so a request touching any number of feature views costs **1** query on both the sync and async paths. This helps most where round trips dominate — a handful of entities against a remote database. Past `MAX_BATCHED_READ_KEYS` entity keys in a request the store reverts to one query per view, because at that size the saved round trips no longer offset holding every view's rows at once. As with Redis, consolidating feature views still reduces application-layer overhead.
36
38
{% endhint %}
37
39
38
40
### Feature services are free (and can be faster)
@@ -87,7 +89,7 @@ Requesting just `combined_score` triggers reads from **both** `driver_stats_fv`
87
89
88
90
## Pre-computed feature vectors
89
91
90
-
When a `get_online_features()` request touches multiple feature views, the server issues a separate store read per feature view. For services spanning 5–15+ feature views, this fan-out dominates latency — even with Redis pipeline batching, the protobuf deserialization and response-building overhead grows linearly with the number of views.
92
+
When a `get_online_features()` request touches multiple feature views, the server issues a separate store read per feature view. For services spanning 5–15+ feature views, this fan-out dominates latency — even where the store batches its reads (Redis, PostgreSQL), the protobuf deserialization and response-building overhead grows linearly with the number of views.
91
93
92
94
**Pre-computed feature vectors** solve this by storing all of a feature service's features for each entity as a single serialized blob. At read time, the server fetches one blob per entity instead of N reads per feature view, reducing the operation to O(1).
93
95
@@ -276,7 +278,7 @@ The online store is the single largest factor in `get_online_features()` latency
| **Redis / Dragonfly** | < 1 ms | No (threadpool) | Ultra-low latency, high throughput; all FV reads batched into 1 pipeline | Requires in-memory capacity for your dataset |
278
280
| **DynamoDB** | 2–5 ms | Yes | Serverless, auto-scaling on AWS | Pay-per-request cost; batch API limits (100 items) |
279
-
| **PostgreSQL** | 3–10 ms | No (threadpool) | Teams with existing Postgres infra | Connection pooling needed at scale |
281
+
| **PostgreSQL** | 3–10 ms | No (threadpool) | Teams with existing Postgres infra; all FV reads batched into 1 query | Connection pooling needed at scale |
280
282
| **MongoDB** | 2–5 ms | Yes | Flexible schema, async-native | Requires index tuning for large datasets |
281
283
| **Aerospike** | < 1 ms | No (threadpool) | Ultra-low latency, hybrid memory (RAM + SSD), large datasets | Namespace must be pre-configured on the cluster |
282
284
| **Bigtable** | 3–8 ms | No (threadpool) | Large-scale GCP workloads | Row-key design affects read performance |
@@ -304,8 +306,8 @@ The feature server can read from the online store using either an **async** or *
| **PostgreSQL** | Implemented | No | Has `online_read_async` but does not yet advertise via `async_supported`; uses sync/threadpool path |
308
-
| **Redis** | Implemented | **Yes** | `online_read_async` and `online_write_batch_async` both implemented; uses sync/threadpool path for `get_online_features` (overridden with batched single pipeline) |
309
+
| **PostgreSQL** | Implemented | No | Has `online_read_async` but does not yet advertise via `async_supported`; uses sync/threadpool path. `_read_features_per_fv` is overridden to batch all feature view reads into a single `UNION ALL` query |
310
+
| **Redis** | Implemented | **Yes** | `online_read_async` and `online_write_batch_async` both implemented; uses sync/threadpool path for `get_online_features` (`_read_features_per_fv` overridden with batched single pipeline) |
309
311
| **Aerospike** | Implemented | No | Async methods wrap the blocking C client via `run_in_executor`; does not yet advertise via `async_supported`, so the server still uses the threadpool path |
310
312
| All others | No | No | Fall back to sync with `run_in_threadpool()` |
311
313
@@ -387,7 +389,7 @@ online_store:
387
389
388
390
#### Batched multi-feature-view reads
389
391
390
-
The Redis online store overrides `get_online_features()` to issue all `HMGET` commands — across every feature view in the request — in a **single pipeline execution**. This reduces Redis round trips from `N` (one per feature view) to `1` regardless of request size.
392
+
The Redis online store overrides `_read_features_per_fv()` to issue all `HMGET` commands — across every feature view in the request — in a **single pipeline execution**. This reduces Redis round trips from `N` (one per feature view) to `1` regardless of request size.
0 commit comments