Skip to content

Commit d2c15f8

Browse files
committed
docs: Describe the on demand feature views added to the local template
Update the quickstart to include transformed_conv_rate_python and transformed_conv_rate_on_write in the feature_definitions.py listing and in the feast apply / materialize outputs, and add a step contrasting read-time (Pandas / native Python) and write-time transformations. Also list the on demand feature views in the template README. Part of feast-dev#5478 Signed-off-by: ryota717 <[email protected]>
1 parent 736e005 commit d2c15f8

2 files changed

Lines changed: 142 additions & 5 deletions

File tree

‎docs/getting-started/quickstart.md‎

Lines changed: 137 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,7 @@ entity_key_serialization_version: 3
122122
# This is an example feature definition file
123123

124124
from datetime import timedelta
125+
from typing import Any
125126

126127
import pandas as pd
127128

@@ -194,7 +195,8 @@ input_request = RequestSource(
194195

195196

196197
# Define an on demand feature view which can generate new features based on
197-
# existing feature views and RequestSource features
198+
# existing feature views and RequestSource features. The transformation runs at
199+
# read time, every time the features are requested. Pandas mode is the default.
198200
@on_demand_feature_view(
199201
sources=[driver_stats_fv, input_request],
200202
schema=[
@@ -209,6 +211,36 @@ def transformed_conv_rate(inputs: pd.DataFrame) -> pd.DataFrame:
209211
return df
210212

211213

214+
# The same transformation written in native Python mode (mode="python"). The UDF
215+
# receives a dict mapping each input feature name to a list of values (one per
216+
# row) and returns a dict with the same shape.
217+
#
218+
# Only the features the UDF needs are selected from the source feature view.
219+
# This is required here: driver_stats_fv also has Map / Struct / Json fields, and
220+
# Python mode feature inference cannot generate sample values for those types.
221+
@on_demand_feature_view(
222+
sources=[driver_stats_fv[["conv_rate"]], input_request],
223+
schema=[
224+
Field(name="conv_rate_plus_val1_python", dtype=Float64),
225+
Field(name="conv_rate_plus_val2_python", dtype=Float64),
226+
],
227+
mode="python",
228+
)
229+
def transformed_conv_rate_python(inputs: dict[str, Any]) -> dict[str, Any]:
230+
return {
231+
"conv_rate_plus_val1_python": [
232+
conv_rate + val_to_add
233+
for conv_rate, val_to_add in zip(inputs["conv_rate"], inputs["val_to_add"])
234+
],
235+
"conv_rate_plus_val2_python": [
236+
conv_rate + val_to_add_2
237+
for conv_rate, val_to_add_2 in zip(
238+
inputs["conv_rate"], inputs["val_to_add_2"]
239+
)
240+
],
241+
}
242+
243+
212244
# This groups features into a model version
213245
driver_activity_v1 = FeatureService(
214246
name="driver_activity_v1",
@@ -265,6 +297,32 @@ driver_activity_v3 = FeatureService(
265297
name="driver_activity_v3",
266298
features=[driver_stats_fresh_fv, transformed_conv_rate_fresh],
267299
)
300+
301+
302+
# Setting write_to_online_store=True runs the transformation at write time instead:
303+
# the derived features are computed once when data is materialized or written to
304+
# the online store, and are then served like any other pre-computed feature.
305+
# Because the results are persisted, the view must declare its entities and can
306+
# only depend on other feature views (not on request-time data).
307+
@on_demand_feature_view(
308+
entities=[driver],
309+
sources=[driver_stats_fv[["conv_rate", "acc_rate", "avg_daily_trips"]]],
310+
schema=[
311+
Field(name="conv_rate_x_acc_rate", dtype=Float64),
312+
Field(name="expected_daily_conversions", dtype=Float64),
313+
],
314+
mode="pandas",
315+
write_to_online_store=True,
316+
)
317+
def transformed_conv_rate_on_write(inputs: pd.DataFrame) -> pd.DataFrame:
318+
df = pd.DataFrame()
319+
df["conv_rate_x_acc_rate"] = (inputs["conv_rate"] * inputs["acc_rate"]).astype(
320+
"float64"
321+
)
322+
df["expected_daily_conversions"] = (
323+
inputs["avg_daily_trips"] * inputs["conv_rate"]
324+
).astype("float64")
325+
return df
268326
```
269327
{% endtab %}
270328
{% endtabs %}
@@ -330,13 +388,16 @@ Created entity driver
330388
Created feature view driver_hourly_stats
331389
Created feature view driver_hourly_stats_fresh
332390
Created on demand feature view transformed_conv_rate
391+
Created on demand feature view transformed_conv_rate_python
333392
Created on demand feature view transformed_conv_rate_fresh
393+
Created on demand feature view transformed_conv_rate_on_write
334394
Created feature service driver_activity_v3
335395
Created feature service driver_activity_v1
336396
Created feature service driver_activity_v2
337397

338398
Created sqlite table my_project_driver_hourly_stats_fresh
339399
Created sqlite table my_project_driver_hourly_stats
400+
Created sqlite table my_project_transformed_conv_rate_on_write
340401
```
341402
{% endtab %}
342403
{% endtabs %}
@@ -503,6 +564,8 @@ print(training_df.head())
503564

504565
We now serialize the latest values of features from the materialization window to prepare for serving. `materialize-incremental` starts from each feature view's most recent materialization end date. On the first run, it starts from the current time minus the feature view's `ttl`. In this example, the initial lookback is one day (`ttl` was set on the `FeatureView` instances in `feature_definitions.py`).
505566

567+
On demand feature views with `write_to_online_store=True` (here `transformed_conv_rate_on_write`) are materialized too: their source features are read from the offline store, the transformation is applied once, and the results are written to the online store.
568+
506569
{% tabs %}
507570
{% tab title="Bash (with timestamp)" %}
508571
```bash
@@ -522,13 +585,13 @@ feast materialize --disable-event-timestamp
522585
{% tabs %}
523586
{% tab title="Output" %}
524587
```bash
525-
Materializing 2 feature views to 2024-04-19 10:59:58-04:00 into the sqlite online store.
588+
Materializing 3 feature views to 2024-04-19 10:59:58-04:00 into the sqlite online store.
526589

527590
driver_hourly_stats from 2024-04-18 15:00:46-04:00 to 2024-04-19 10:59:58-04:00:
528591
100%|████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 370.32it/s]
529592
driver_hourly_stats_fresh from 2024-04-18 15:00:46-04:00 to 2024-04-19 10:59:58-04:00:
530593
100%|███████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 1046.64it/s]
531-
Materializing 2 feature views to 2024-04-19 10:59:58-04:00 into the sqlite online store.
594+
transformed_conv_rate_on_write:
532595
```
533596
{% endtab %}
534597
{% endtabs %}
@@ -628,7 +691,76 @@ pprint(feature_vector)
628691
{% endtab %}
629692
{% endtabs %}
630693

631-
## Step 9: Browse your features with the Web UI (experimental)
694+
### Step 9: Transforming features on read vs. on write
695+
696+
The template defines on demand feature views in both flavors. `transformed_conv_rate` (Pandas mode) and
697+
`transformed_conv_rate_python` (native Python mode) run their transformation at read time, so they can combine
698+
stored features with request-time data such as `val_to_add` and `val_to_add_2`. Both produce the same values:
699+
700+
{% tabs %}
701+
{% tab title="Python" %}
702+
```python
703+
feature_vector = store.get_online_features(
704+
features=[
705+
"transformed_conv_rate:conv_rate_plus_val1",
706+
"transformed_conv_rate_python:conv_rate_plus_val1_python",
707+
],
708+
entity_rows=[
709+
{"driver_id": 1004, "val_to_add": 1000, "val_to_add_2": 2000},
710+
{"driver_id": 1005, "val_to_add": 1001, "val_to_add_2": 2002},
711+
],
712+
).to_dict()
713+
pprint(feature_vector)
714+
```
715+
{% endtab %}
716+
{% endtabs %}
717+
718+
{% tabs %}
719+
{% tab title="Output" %}
720+
```bash
721+
{
722+
'conv_rate_plus_val1': [1000.0459796078503, 1001.811998963356],
723+
'conv_rate_plus_val1_python': [1000.0459796078503, 1001.811998963356],
724+
'driver_id': [1004, 1005]
725+
}
726+
```
727+
{% endtab %}
728+
{% endtabs %}
729+
730+
`transformed_conv_rate_on_write` instead has `write_to_online_store=True`: its transformation already ran during
731+
`materialize-incremental` (and runs again whenever new rows are written with `store.write_to_online_store(...)`), so
732+
reading it is a plain lookup with no transformation at request time:
733+
734+
{% tabs %}
735+
{% tab title="Python" %}
736+
```python
737+
feature_vector = store.get_online_features(
738+
features=[
739+
"transformed_conv_rate_on_write:conv_rate_x_acc_rate",
740+
"transformed_conv_rate_on_write:expected_daily_conversions",
741+
],
742+
entity_rows=[{"driver_id": 1004}, {"driver_id": 1005}],
743+
).to_dict()
744+
pprint(feature_vector)
745+
```
746+
{% endtab %}
747+
{% endtabs %}
748+
749+
{% tabs %}
750+
{% tab title="Output" %}
751+
```bash
752+
{
753+
'conv_rate_x_acc_rate': [0.005327459424734116, 0.4349033236503601],
754+
'driver_id': [1004, 1005],
755+
'expected_daily_conversions': [32.231705103069544, 391.3835003376007]
756+
}
757+
```
758+
{% endtab %}
759+
{% endtabs %}
760+
761+
See [On demand feature views](../reference/beta-on-demand-feature-view.md) for more details on the transformation modes and on write-time transformations.
762+
763+
## Step 10: Browse your features with the Web UI (experimental)
632764

633765
View all registered features, data sources, entities, and feature services with the Web UI.
634766

@@ -660,7 +792,7 @@ INFO: Uvicorn running on http://0.0.0.0:8888 (Press CTRL+C to quit)
660792

661793
![](../reference/ui.png)
662794

663-
## Step 10: Re-examine `test_workflow.py`
795+
## Step 11: Re-examine `test_workflow.py`
664796
Take a look at `test_workflow.py` again. It showcases many sample flows on how to interact with Feast. You'll see these
665797
show up in the upcoming concepts + architecture + tutorial pages as well.
666798

‎sdk/python/feast/templates/local/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,11 @@ uses this repo. A quick view of what's in this repository's `feature_repo/` dire
44

55
* `data/` contains raw demo parquet data
66
* `feature_repo/feature_definitions.py` contains demo feature definitions
7+
- `driver_hourly_stats` / `driver_hourly_stats_fresh`: regular feature views backed by a parquet file and a push source
8+
- `transformed_conv_rate` / `transformed_conv_rate_python`: on demand feature views that transform features at
9+
read time, written in Pandas mode and in native Python mode respectively
10+
- `transformed_conv_rate_on_write`: an on demand feature view with `write_to_online_store=True`, whose
11+
transformation runs when data is materialized or written to the online store instead of on every read
712
* `feature_repo/feature_store.yaml` contains a demo setup configuring where data sources are
813
* `feature_repo/test_workflow.py` showcases how to run all key Feast commands, including defining, retrieving, and pushing features.
914

0 commit comments

Comments
 (0)