Skip to content

Commit 7c50ab5

Browse files
adchiaadchia
authored
fix: Fix push API to respect feature view's already inferred entity types (#3172)
* fix: Fix push API to respect feature view's already inferred entity types Signed-off-by: Danny Chiao <[email protected]> * fix roadmap Signed-off-by: Danny Chiao <[email protected]> * fix helm Signed-off-by: Danny Chiao <[email protected]> Signed-off-by: Danny Chiao <[email protected]>
1 parent b4ef834 commit 7c50ab5

9 files changed

Lines changed: 18 additions & 28 deletions

File tree

‎Makefile‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -356,15 +356,11 @@ lint-go: compile-protos-go compile-go-lib
356356

357357
# Docker
358358

359-
build-docker: build-ci-docker build-feature-server-python-docker build-feature-server-python-aws-docker build-feature-transformation-server-docker build-feature-server-java-docker
359+
build-docker: build-feature-server-python-docker build-feature-server-python-aws-docker build-feature-transformation-server-docker build-feature-server-java-docker
360360

361361
push-ci-docker:
362362
docker push $(REGISTRY)/feast-ci:$(VERSION)
363363

364-
# TODO(adchia): consider removing. This doesn't run successfully right now
365-
build-ci-docker:
366-
docker buildx build -t $(REGISTRY)/feast-ci:$(VERSION) -f infra/docker/ci/Dockerfile --load .
367-
368364
push-feature-server-python-docker:
369365
docker push $(REGISTRY)/feature-server:$$VERSION
370366

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -185,8 +185,8 @@ The list below contains the functionality that contributors are planning to deve
185185
* [ ] Batch transformation (In progress. See [RFC](https://docs.google.com/document/d/1964OkzuBljifDvkV-0fakp2uaijnVzdwWNGdz7Vz50A/edit))
186186
* **Streaming**
187187
* [x] [Custom streaming ingestion job support](https://docs.feast.dev/how-to-guides/creating-a-custom-provider)
188-
* [x] [Push based streaming data ingestion to online store (Alpha)](https://docs.feast.dev/reference/data-sources/push)
189-
* [x] [Push based streaming data ingestion to offline store (Alpha)](https://docs.feast.dev/reference/data-sources/push)
188+
* [x] [Push based streaming data ingestion to online store](https://docs.feast.dev/reference/data-sources/push)
189+
* [x] [Push based streaming data ingestion to offline store](https://docs.feast.dev/reference/data-sources/push)
190190
* **Deployments**
191191
* [x] AWS Lambda (Alpha release. See [RFC](https://docs.google.com/document/d/1eZWKWzfBif66LDN32IajpaG-j82LSHCCOzY6R7Ax7MI/edit))
192192
* [x] Kubernetes (See [guide](https://docs.feast.dev/how-to-guides/running-feast-in-production#4.3.-java-based-feature-server-deployed-on-kubernetes))
@@ -202,7 +202,7 @@ The list below contains the functionality that contributors are planning to deve
202202
* [x] Model-centric feature tracking (feature services)
203203
* [x] Amundsen integration (see [Feast extractor](https://github.com/amundsen-io/amundsen/blob/main/databuilder/databuilder/extractor/feast_extractor.py))
204204
* [x] DataHub integration (see [DataHub Feast docs](https://datahubproject.io/docs/generated/ingestion/sources/feast/))
205-
* [x] Feast Web UI (Alpha release. See [docs](https://docs.feast.dev/reference/alpha-web-ui))
205+
* [x] Feast Web UI (Beta release. See [docs](https://docs.feast.dev/reference/alpha-web-ui))
206206

207207

208208
## 🎓 Important Resources

‎docs/reference/data-sources/push.md‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
11
# Push source
22

3-
**Warning**: This is an _experimental_ feature. It's intended for early testing and feedback, and could change without warnings in future releases.
4-
53
## Description
64

75
Push sources allow feature values to be pushed to the online store and offline store in real time. This allows fresh feature values to be made available to applications. Push sources supercede the

‎docs/roadmap.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -43,15 +43,16 @@ The list below contains the functionality that contributors are planning to deve
4343
* [ ] Batch transformation (In progress. See [RFC](https://docs.google.com/document/d/1964OkzuBljifDvkV-0fakp2uaijnVzdwWNGdz7Vz50A/edit))
4444
* **Streaming**
4545
* [x] [Custom streaming ingestion job support](https://docs.feast.dev/how-to-guides/creating-a-custom-provider)
46-
* [x] [Push based streaming data ingestion to online store (Alpha)](https://docs.feast.dev/reference/data-sources/push)
47-
* [x] [Push based streaming data ingestion to offline store (Alpha)](https://docs.feast.dev/reference/data-sources/push)
46+
* [x] [Push based streaming data ingestion to online store](https://docs.feast.dev/reference/data-sources/push)
47+
* [x] [Push based streaming data ingestion to offline store](https://docs.feast.dev/reference/data-sources/push)
4848
* **Deployments**
4949
* [x] AWS Lambda (Alpha release. See [RFC](https://docs.google.com/document/d/1eZWKWzfBif66LDN32IajpaG-j82LSHCCOzY6R7Ax7MI/edit))
5050
* [x] Kubernetes (See [guide](https://docs.feast.dev/how-to-guides/running-feast-in-production#4.3.-java-based-feature-server-deployed-on-kubernetes))
5151
* **Feature Serving**
5252
* [x] Python Client
5353
* [x] [Python feature server](https://docs.feast.dev/reference/feature-servers/python-feature-server)
54-
* [x] [Go feature server](https://docs.feast.dev/reference/feature-servers/go-feature-server)
54+
* [x] [Java feature server (alpha)](https://github.com/feast-dev/feast/blob/master/infra/charts/feast/README.md)
55+
* [x] [Go feature server (alpha)](https://docs.feast.dev/reference/feature-servers/go-feature-server)
5556
* **Data Quality Management (See [RFC](https://docs.google.com/document/d/110F72d4NTv80p35wDSONxhhPBqWRwbZXG4f9mNEMd98/edit))**
5657
* [x] Data profiling and validation (Great Expectations)
5758
* **Feature Discovery and Governance**
@@ -60,4 +61,4 @@ The list below contains the functionality that contributors are planning to deve
6061
* [x] Model-centric feature tracking (feature services)
6162
* [x] Amundsen integration (see [Feast extractor](https://github.com/amundsen-io/amundsen/blob/main/databuilder/databuilder/extractor/feast_extractor.py))
6263
* [x] DataHub integration (see [DataHub Feast docs](https://datahubproject.io/docs/generated/ingestion/sources/feast/))
63-
* [x] Feast Web UI (Alpha release. See [docs](https://docs.feast.dev/reference/alpha-web-ui))
64+
* [x] Feast Web UI (Beta release. See [docs](https://docs.feast.dev/reference/alpha-web-ui))

‎infra/charts/feast/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
# Feast Helm Charts
1+
# Feast Java Helm Charts (alpha)
22

3-
This repo contains Helm charts for Feast components that are being installed on Kubernetes:
3+
This repo contains Helm charts for Feast Java components that are being installed on Kubernetes:
44
* Feast (root chart): The complete Helm chart containing all Feast components and dependencies. Most users will use this chart, but can selectively enable/disable subcharts using the values.yaml file.
55
* [Feature Server](charts/feature-server): High performant JVM-based implementation of feature server.
66
* [Transformation Service](charts/transformation-service): Transformation server for calculating on-demand features

‎infra/charts/feast/README.md.gotmpl‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
# Feast Helm Charts
1+
# Feast Java Helm Charts (alpha)
22

3-
This repo contains Helm charts for Feast components that are being installed on Kubernetes:
3+
This repo contains Helm charts for Feast Java components that are being installed on Kubernetes:
44
* Feast (root chart): The complete Helm chart containing all Feast components and dependencies. Most users will use this chart, but can selectively enable/disable subcharts using the values.yaml file.
55
* [Feature Server](charts/feature-server): High performant JVM-based implementation of feature server.
66
* [Transformation Service](charts/transformation-service): Transformation server for calculating on-demand features

‎sdk/python/feast/feature_store.py‎

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1478,13 +1478,8 @@ def write_to_online_store(
14781478
feature_view = self.get_feature_view(
14791479
feature_view_name, allow_registry_cache=allow_registry_cache
14801480
)
1481-
entities = []
1482-
for entity_name in feature_view.entities:
1483-
entities.append(
1484-
self.get_entity(entity_name, allow_registry_cache=allow_registry_cache)
1485-
)
14861481
provider = self._get_provider()
1487-
provider.ingest_df(feature_view, entities, df)
1482+
provider.ingest_df(feature_view, df)
14881483

14891484
@log_exceptions_and_usage
14901485
def write_to_offline_store(

‎sdk/python/feast/infra/passthrough_provider.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -193,7 +193,6 @@ def online_read(
193193
def ingest_df(
194194
self,
195195
feature_view: FeatureView,
196-
entities: List[Entity],
197196
df: pd.DataFrame,
198197
):
199198
set_usage_attribute("provider", self.__class__.__name__)
@@ -204,7 +203,10 @@ def ingest_df(
204203
table, feature_view.batch_source.field_mapping
205204
)
206205

207-
join_keys = {entity.join_key: entity.value_type for entity in entities}
206+
join_keys = {
207+
entity.name: entity.dtype.to_value_type()
208+
for entity in feature_view.entity_columns
209+
}
208210
rows_to_write = _convert_arrow_to_proto(table, feature_view, join_keys)
209211

210212
self.online_write_batch(

‎sdk/python/feast/infra/provider.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -123,15 +123,13 @@ def online_write_batch(
123123
def ingest_df(
124124
self,
125125
feature_view: FeatureView,
126-
entities: List[Entity],
127126
df: pd.DataFrame,
128127
):
129128
"""
130129
Persists a dataframe to the online store.
131130
132131
Args:
133132
feature_view: The feature view to which the dataframe corresponds.
134-
entities: The entities that are referenced by the dataframe.
135133
df: The dataframe to be persisted.
136134
"""
137135
pass

0 commit comments

Comments
 (0)