Skip to content

Commit a0d985d

Browse files
DKilkennyntkathole
authored andcommitted
feat: Batch Postgres online reads across feature views
The generic OnlineStore._read_features_per_fv loop issues one online_read per feature view, so a request spanning N feature views costs N round trips. Redis already overrides the seam to batch; Postgres now does too. Each feature view has its own table, so the per-view reads are combined with UNION ALL and split apart again by a constant tag column. The tag is what keeps them separable: the table name is not in the result set, and two feature views can legitimately return the same entity key AND the same feature name. Batching only pays where round trips dominate. Measured against Postgres 16 with a 2ms round trip, a 10-view request is ~5x faster at 1 entity and ~3x at 10, but a wash at 5000 entities, where holding every view's rows at once costs tens of MB per in-flight request. Requests above MAX_BATCHED_READ_KEYS entity keys therefore fall back to the generic per-view path, which processes and releases one view at a time. The threshold is checked from the request shape before any keys are serialized, so choosing not to batch costs nothing. Both the sync and async paths are overridden. The generic async path already runs the per-view queries concurrently via asyncio.gather, so the async win is N concurrent queries becoming one rather than a change from serial. Note that Postgres does not advertise async_supported today, so the async path is not yet reachable from the feature server. Also corrects four doc references that described Redis and Postgres as overriding get_online_features; both override _read_features_per_fv. Refs feast-dev#3259 Signed-off-by: DKilkenny <[email protected]>
1 parent 1118d3e commit a0d985d

4 files changed

Lines changed: 893 additions & 7 deletions

File tree

‎docs/how-to-guides/online-server-performance-tuning.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,9 @@ When the server processes a `get_online_features()` call, it groups the requeste
3232
**Guideline:** For features that share the same entity key and are frequently requested together, consolidate them into a **single feature view**. This reduces the number of store round-trips per request. Split feature views only when features have different entities, different materialization schedules, or different data source update frequencies.
3333

3434
{% hint style="info" %}
35-
**Redis exception:** The Redis online store overrides `get_online_features()` to batch all `HMGET` commands across every feature view into a **single pipeline execution**. Because all feature views for the same entity share one Redis hash key, the number of Redis round trips is always **1**, regardless of how many feature views the request touches. This means the "fewer feature views" guideline is less critical for Redis than for other stores — but consolidating feature views still reduces serialization and protobuf overhead at the application layer.
35+
**Redis exception:** The Redis online store overrides `_read_features_per_fv()` to batch all `HMGET` commands across every feature view into a **single pipeline execution**. Because all feature views for the same entity share one Redis hash key, the number of Redis round trips is always **1**, regardless of how many feature views the request touches. This means the "fewer feature views" guideline is less critical for Redis than for other stores — but consolidating feature views still reduces serialization and protobuf overhead at the application layer.
36+
37+
**PostgreSQL exception:** PostgreSQL stores each feature view in its own table and overrides `_read_features_per_fv()` to combine the per-view reads into a **single `UNION ALL` statement**, so a request touching any number of feature views costs **1** query on both the sync and async paths. This helps most where round trips dominate — a handful of entities against a remote database. Past `MAX_BATCHED_READ_KEYS` entity keys in a request the store reverts to one query per view, because at that size the saved round trips no longer offset holding every view's rows at once. As with Redis, consolidating feature views still reduces application-layer overhead.
3638
{% endhint %}
3739

3840
### Feature services are free (and can be faster)
@@ -87,7 +89,7 @@ Requesting just `combined_score` triggers reads from **both** `driver_stats_fv`
8789

8890
## Pre-computed feature vectors
8991

90-
When a `get_online_features()` request touches multiple feature views, the server issues a separate store read per feature view. For services spanning 5–15+ feature views, this fan-out dominates latency — even with Redis pipeline batching, the protobuf deserialization and response-building overhead grows linearly with the number of views.
92+
When a `get_online_features()` request touches multiple feature views, the server issues a separate store read per feature view. For services spanning 5–15+ feature views, this fan-out dominates latency — even where the store batches its reads (Redis, PostgreSQL), the protobuf deserialization and response-building overhead grows linearly with the number of views.
9193

9294
**Pre-computed feature vectors** solve this by storing all of a feature service's features for each entity as a single serialized blob. At read time, the server fetches one blob per entity instead of N reads per feature view, reducing the operation to O(1).
9395

@@ -276,7 +278,7 @@ The online store is the single largest factor in `get_online_features()` latency
276278
| ----- | ------------------- | ---------- | -------- | ------------- |
277279
| **Redis / Dragonfly** | < 1 ms | No (threadpool) | Ultra-low latency, high throughput; all FV reads batched into 1 pipeline | Requires in-memory capacity for your dataset |
278280
| **DynamoDB** | 2–5 ms | Yes | Serverless, auto-scaling on AWS | Pay-per-request cost; batch API limits (100 items) |
279-
| **PostgreSQL** | 3–10 ms | No (threadpool) | Teams with existing Postgres infra | Connection pooling needed at scale |
281+
| **PostgreSQL** | 3–10 ms | No (threadpool) | Teams with existing Postgres infra; all FV reads batched into 1 query | Connection pooling needed at scale |
280282
| **MongoDB** | 2–5 ms | Yes | Flexible schema, async-native | Requires index tuning for large datasets |
281283
| **Aerospike** | < 1 ms | No (threadpool) | Ultra-low latency, hybrid memory (RAM + SSD), large datasets | Namespace must be pre-configured on the cluster |
282284
| **Bigtable** | 3–8 ms | No (threadpool) | Large-scale GCP workloads | Row-key design affects read performance |
@@ -304,8 +306,8 @@ The feature server can read from the online store using either an **async** or *
304306
| ----- | ---------- | ----------- | ----- |
305307
| **DynamoDB** | Yes | Yes | Uses `aiobotocore` for non-blocking I/O |
306308
| **MongoDB** | Yes | Yes | Uses `motor` (async MongoDB driver) |
307-
| **PostgreSQL** | Implemented | No | Has `online_read_async` but does not yet advertise via `async_supported`; uses sync/threadpool path |
308-
| **Redis** | Implemented | **Yes** | `online_read_async` and `online_write_batch_async` both implemented; uses sync/threadpool path for `get_online_features` (overridden with batched single pipeline) |
309+
| **PostgreSQL** | Implemented | No | Has `online_read_async` but does not yet advertise via `async_supported`; uses sync/threadpool path. `_read_features_per_fv` is overridden to batch all feature view reads into a single `UNION ALL` query |
310+
| **Redis** | Implemented | **Yes** | `online_read_async` and `online_write_batch_async` both implemented; uses sync/threadpool path for `get_online_features` (`_read_features_per_fv` overridden with batched single pipeline) |
309311
| **Aerospike** | Implemented | No | Async methods wrap the blocking C client via `run_in_executor`; does not yet advertise via `async_supported`, so the server still uses the threadpool path |
310312
| All others | No | No | Fall back to sync with `run_in_threadpool()` |
311313

@@ -387,7 +389,7 @@ online_store:
387389

388390
#### Batched multi-feature-view reads
389391

390-
The Redis online store overrides `get_online_features()` to issue all `HMGET` commands — across every feature view in the request — in a **single pipeline execution**. This reduces Redis round trips from `N` (one per feature view) to `1` regardless of request size.
392+
The Redis online store overrides `_read_features_per_fv()` to issue all `HMGET` commands — across every feature view in the request — in a **single pipeline execution**. This reduces Redis round trips from `N` (one per feature view) to `1` regardless of request size.
391393

392394
| Feature views | Round trips (other stores) | Round trips (Redis) |
393395
| :---: | :---: | :---: |

‎sdk/python/feast/infra/online_stores/postgres_online_store/postgres.py‎

Lines changed: 243 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
Generator,
1111
List,
1212
Literal,
13+
NamedTuple,
1314
Optional,
1415
Sequence,
1516
Tuple,
@@ -20,7 +21,7 @@
2021
from psycopg.connection import Connection
2122
from psycopg_pool import AsyncConnectionPool, ConnectionPool
2223

23-
from feast import Entity, FeatureView, ValueType
24+
from feast import Entity, FeatureView, ValueType, utils
2425
from feast.filter_models import (
2526
ComparisonFilter,
2627
CompoundFilter,
@@ -55,6 +56,15 @@
5556
"inner_product": "<#>",
5657
}
5758

59+
# Above this many entity keys in a request, batching stops paying for itself: the
60+
# saved round trips are amortized away by the payload, while holding every feature
61+
# view's rows at once makes peak memory grow with the number of views. Measured
62+
# against Postgres 16, a 10-view request is ~3-5x faster at 1-100 entities and a
63+
# wash at 5000, where the combined result costs tens of MB per in-flight request.
64+
# Past the threshold the generic one-query-per-view path is used instead, which
65+
# processes and releases a single view at a time.
66+
MAX_BATCHED_READ_KEYS = 2048
67+
5868
_PG_COMPARISON_OPS: Dict[str, str] = {
5969
"eq": "=",
6070
"ne": "!=",
@@ -65,6 +75,16 @@
6575
}
6676

6777

78+
class _BatchedRead(NamedTuple):
79+
"""One feature view's share of a batched online read."""
80+
81+
table: FeatureView
82+
requested_features: List[str]
83+
keys: List[bytes]
84+
idxs: Tuple[List[int], ...]
85+
output_len: int
86+
87+
6888
class PostgresFilterTranslator(FilterTranslator):
6989
"""Translates Feast filters into Postgres SQL WHERE clause fragments."""
7090

@@ -424,6 +444,13 @@ def _process_rows(
424444
row[0] if isinstance(row[0], bytes) else row[0].tobytes()
425445
].append(row[1:])
426446

447+
return PostgreSQLOnlineStore._result_from_values_dict(keys, values_dict)
448+
449+
@staticmethod
450+
def _result_from_values_dict(
451+
keys: List[bytes], values_dict: Dict[bytes, List[Tuple]]
452+
) -> List[Tuple[Optional[datetime], Optional[Dict[str, ValueProto]]]]:
453+
"""Assemble per-entity rows in ``keys`` order from a feature-value mapping."""
427454
result: List[Tuple[Optional[datetime], Optional[Dict[str, ValueProto]]]] = []
428455
for key in keys:
429456
if key in values_dict:
@@ -438,6 +465,221 @@ def _process_rows(
438465
result.append((None, None))
439466
return result
440467

468+
def _read_features_per_fv(
469+
self,
470+
config: RepoConfig,
471+
grouped_refs: List,
472+
join_key_values: Dict,
473+
entity_name_to_join_key_map: Dict,
474+
online_features_response,
475+
full_feature_names: bool,
476+
include_feature_view_version_metadata: bool,
477+
) -> None:
478+
"""Read every requested feature view in a single round trip.
479+
480+
The generic path issues one query per feature view. Each feature view lives
481+
in its own table here, so the per-view reads can be combined with UNION ALL
482+
and split apart again afterwards.
483+
"""
484+
if self._too_large_to_batch(grouped_refs, join_key_values):
485+
return super()._read_features_per_fv(
486+
config,
487+
grouped_refs,
488+
join_key_values,
489+
entity_name_to_join_key_map,
490+
online_features_response,
491+
full_feature_names,
492+
include_feature_view_version_metadata,
493+
)
494+
495+
reads = self._prepare_batched_reads(
496+
config, grouped_refs, join_key_values, entity_name_to_join_key_map
497+
)
498+
if not reads:
499+
return
500+
501+
query, params = self._construct_batched_query_and_params(config, reads)
502+
with self._get_conn(config, autocommit=True) as conn, conn.cursor() as cur:
503+
cur.execute(query, params)
504+
buckets = self._bucket_rows(reads, cur)
505+
506+
self._populate_from_buckets(
507+
reads,
508+
buckets,
509+
online_features_response,
510+
full_feature_names,
511+
include_feature_view_version_metadata,
512+
)
513+
514+
async def _read_features_per_fv_async(
515+
self,
516+
config: RepoConfig,
517+
grouped_refs: List,
518+
join_key_values: Dict,
519+
entity_name_to_join_key_map: Dict,
520+
online_features_response,
521+
full_feature_names: bool,
522+
include_feature_view_version_metadata: bool,
523+
) -> None:
524+
"""Async version of :meth:`_read_features_per_fv`.
525+
526+
The generic async path issues the per-view queries concurrently. Combining
527+
them still replaces those N queries with one.
528+
"""
529+
if self._too_large_to_batch(grouped_refs, join_key_values):
530+
return await super()._read_features_per_fv_async(
531+
config,
532+
grouped_refs,
533+
join_key_values,
534+
entity_name_to_join_key_map,
535+
online_features_response,
536+
full_feature_names,
537+
include_feature_view_version_metadata,
538+
)
539+
540+
reads = self._prepare_batched_reads(
541+
config, grouped_refs, join_key_values, entity_name_to_join_key_map
542+
)
543+
if not reads:
544+
return
545+
546+
query, params = self._construct_batched_query_and_params(config, reads)
547+
async with self._get_conn_async(config, autocommit=True) as conn:
548+
async with conn.cursor() as cur:
549+
await cur.execute(query, params)
550+
buckets = [defaultdict(list) for _ in reads] # type: List[Dict[bytes, List[Tuple]]]
551+
async for row in cur:
552+
self._bucket_row(buckets, row)
553+
554+
self._populate_from_buckets(
555+
reads,
556+
buckets,
557+
online_features_response,
558+
full_feature_names,
559+
include_feature_view_version_metadata,
560+
)
561+
562+
def _prepare_batched_reads(
563+
self,
564+
config: RepoConfig,
565+
grouped_refs: List,
566+
join_key_values: Dict,
567+
entity_name_to_join_key_map: Dict,
568+
) -> List["_BatchedRead"]:
569+
"""Resolve the entity keys each feature view needs before querying.
570+
571+
Feature views can be keyed on different entities, so the serialized keys are
572+
computed per view rather than shared.
573+
"""
574+
reads = []
575+
for table, requested_features in grouped_refs:
576+
table_entity_values, idxs, output_len = utils._get_unique_entities(
577+
table, join_key_values, entity_name_to_join_key_map
578+
)
579+
entity_key_protos = utils._get_entity_key_protos(table_entity_values)
580+
reads.append(
581+
_BatchedRead(
582+
table=table,
583+
requested_features=requested_features,
584+
keys=self._prepare_keys(
585+
entity_key_protos, config.entity_key_serialization_version
586+
),
587+
idxs=idxs,
588+
output_len=output_len,
589+
)
590+
)
591+
return reads
592+
593+
@staticmethod
594+
def _construct_batched_query_and_params(
595+
config: RepoConfig, reads: List["_BatchedRead"]
596+
) -> Tuple[sql.Composed, List[Any]]:
597+
"""UNION ALL the per feature view reads into one statement.
598+
599+
Each branch selects a constant tag so the combined result can be split back
600+
apart. The table name itself is not in the result set, and two feature views
601+
can return the same entity key, so the tag is what keeps them separable.
602+
"""
603+
versioning = config.registry.enable_online_feature_view_versioning
604+
branches: List[sql.Composed] = []
605+
params: List[Any] = []
606+
for tag, read in enumerate(reads):
607+
table_name = _table_id(config.project, read.table, versioning)
608+
if read.requested_features:
609+
branch = sql.SQL(
610+
"SELECT {tag} AS fv_tag, entity_key, feature_name, value, event_ts "
611+
"FROM {table} WHERE entity_key = ANY(%s) AND feature_name = ANY(%s)"
612+
).format(tag=sql.Literal(tag), table=sql.Identifier(table_name))
613+
params.extend([read.keys, list(read.requested_features)])
614+
else:
615+
branch = sql.SQL(
616+
"SELECT {tag} AS fv_tag, entity_key, feature_name, value, event_ts "
617+
"FROM {table} WHERE entity_key = ANY(%s)"
618+
).format(tag=sql.Literal(tag), table=sql.Identifier(table_name))
619+
params.append(read.keys)
620+
branches.append(branch)
621+
622+
return sql.SQL(" UNION ALL ").join(branches), params
623+
624+
@staticmethod
625+
def _too_large_to_batch(grouped_refs: List, join_key_values: Dict) -> bool:
626+
"""Whether this request is big enough that batching would cost more than it saves.
627+
628+
Estimated from the request shape rather than from the resolved keys, so that
629+
deciding against batching costs nothing. The number of entity rows in the
630+
request is an upper bound on the unique keys any one view will read.
631+
See :data:`MAX_BATCHED_READ_KEYS`.
632+
"""
633+
entity_rows = max(
634+
(len(values) for values in join_key_values.values()), default=0
635+
)
636+
return entity_rows * len(grouped_refs) > MAX_BATCHED_READ_KEYS
637+
638+
@staticmethod
639+
def _bucket_row(buckets: List[Dict[bytes, List[Tuple]]], row: Tuple) -> None:
640+
"""File one row under its feature view's bucket, keyed by entity key.
641+
642+
Only the three payload columns are kept. Slicing the row instead would copy
643+
it a second time while the combined result is still referenced.
644+
"""
645+
entity_key = row[1] if isinstance(row[1], bytes) else row[1].tobytes()
646+
buckets[row[0]][entity_key].append((row[2], row[3], row[4]))
647+
648+
def _bucket_rows(
649+
self, reads: List["_BatchedRead"], row_iter
650+
) -> List[Dict[bytes, List[Tuple]]]:
651+
"""Consume the combined result one row at a time.
652+
653+
The result spans every requested feature view, so calling ``fetchall()`` and
654+
then slicing each row would hold two or three copies of it at peak. Bucketing
655+
as rows arrive keeps a single copy.
656+
"""
657+
buckets: List[Dict[bytes, List[Tuple]]] = [defaultdict(list) for _ in reads]
658+
for row in row_iter:
659+
self._bucket_row(buckets, row)
660+
return buckets
661+
662+
def _populate_from_buckets(
663+
self,
664+
reads: List["_BatchedRead"],
665+
buckets: List[Dict[bytes, List[Tuple]]],
666+
online_features_response,
667+
full_feature_names: bool,
668+
include_feature_view_version_metadata: bool,
669+
) -> None:
670+
"""Hand each feature view's bucket to the response in grouped_refs order."""
671+
for read, bucket in zip(reads, buckets):
672+
utils._populate_response_from_feature_data(
673+
read.requested_features,
674+
self._result_from_values_dict(read.keys, bucket),
675+
read.idxs,
676+
online_features_response,
677+
full_feature_names,
678+
read.table,
679+
read.output_len,
680+
include_feature_view_version_metadata,
681+
)
682+
441683
def update(
442684
self,
443685
config: RepoConfig,

0 commit comments

Comments
 (0)