Repository navigation
[Feature] Pre-computed feature vectors for online serving #6185
Description
Summary
Add an opt-in pre-computed feature vector capability to dramatically reduce online serving latency for the Python feature server. Instead of assembling features from multiple feature views at read time, pre-compute and store the complete feature vector as a single serialized blob at write/materialize time.
Motivation
The current get-online-features bottleneck is architectural: each request must resolve feature views from the registry, issue sequential online_read calls per feature view, deserialize individual ValueProto objects per feature, assemble them into a GetOnlineFeaturesResponse protobuf, then convert to JSON via MessageToDict. All of this is O(features × entities) in pure Python.
For latency-sensitive use cases, we need improvement that cannot be achieved by optimizing the existing read path alone.
Proposal
At write time (materialize / push), after writing individual features to the online store as usual, also assemble the complete feature vector for each affected FeatureService and store it as a single serialized blob keyed by (project, entity_key, feature_service_name).
At read time, when the request specifies a FeatureService with pre-computation enabled, bypass the per-feature-view assembly pipeline entirely: perform a single key lookup, deserialize one blob, and return.
Opt-in per FeatureService:
FeatureService(
name="realtime_scoring",
features=[user_fv, txn_fv, product_fv],
precompute_online=True, # new flag, default False
)
Only FeatureServices explicitly marked with precompute_online=True trigger pre-computation. This avoids write amplification for services that don't need it.
For FeatureServices containing ODFVs: pre-compute only the non-ODFV features as a partial vector, then apply ODFV transforms at read time on top of the pre-computed base.
Activity
antznette1 commentedon Mar 28, 2026
I think the bottleneck is indeed architectural. I’d be interested in contributing an opt-in
precomputed vectorpath for the Python feature server.ntkathole commentedon Mar 29, 2026
@franciscojavierarceo thoughts on this?
franciscojavierarceo commentedon Mar 30, 2026
I am generally pro this, I do think that we can improve the multi-table read latency by not processing the reads sequentially obviously, but the bottleneck will always be the longest poll. It is for sure more efficient to pre-compute all featureviews into one feature service table but this gets complicated when handling fault tolerance.
antznette1 commentedon Mar 30, 2026
parallelizing multi-table reads can help, but it only shifts the curve; p99 is still bounded by the slowest online store call and Python-side assembly or deserialization overhead.
antznette1 commentedon Mar 30, 2026
What I’m proposing is an opt-in, “precomputed feature vector” read path that complements (not replaces) parallel reads.
antznette1 commentedon Apr 1, 2026
Can I go ahead and PR this proposal or you have a better solution that I can hop on?
nquinn408 commentedon Apr 16, 2026
@antznette1 , is this like a "materialized view" implementation on top of the online feature store?
ntkathole commentedon Jun 2, 2026
@nquinn408 This is conceptually very similar to a materialized view, the difference from a traditional materialized view is that Feast stores the precomputed vector as a serialized protobuf blob keyed by entity, rather than a relational table.