Skip to content

Commit 564e965

Browse files
haoxu0Copilot
haoxu0
and
Copilot
authored
feat: Support aggregation in odfv (#5666)
* add aggregation to odfv Signed-off-by: hao-xu5 <[email protected]> * add aggregation to odfv Signed-off-by: hao-xu5 <[email protected]> * add aggregation to odfv Signed-off-by: hao-xu5 <[email protected]> * fix linting Signed-off-by: hao-xu5 <[email protected]> * Apply suggestions from code review Co-authored-by: Copilot <[email protected]> * fix linting Signed-off-by: hao-xu5 <[email protected]> * fix linting Signed-off-by: hao-xu5 <[email protected]> * update doc Signed-off-by: hao-xu5 <[email protected]> * update doc Signed-off-by: hao-xu5 <[email protected]> --------- Signed-off-by: hao-xu5 <[email protected]> Co-authored-by: hao-xu5 <[email protected]> Co-authored-by: Copilot <[email protected]>
1 parent c6971d2 commit 564e965

14 files changed

Lines changed: 233 additions & 19 deletions

File tree

‎docs/reference/beta-on-demand-feature-view.md‎

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,10 +35,40 @@ When defining an ODFV, you can specify the transformation mode using the `mode`
3535

3636
### Singleton Transformations in Native Python Mode
3737

38-
Native Python mode supports transformations on singleton dictionaries by setting `singleton=True`. This allows you to
39-
write transformation functions that operate on a single row at a time, making the code more intuitive and aligning with
38+
Native Python mode supports transformations on singleton dictionaries by setting `singleton=True`. This allows you to
39+
write transformation functions that operate on a single row at a time, making the code more intuitive and aligning with
4040
how data scientists typically think about data transformations.
4141

42+
## Aggregations
43+
44+
On Demand Feature Views support aggregations that compute aggregate statistics over groups of rows. When using aggregations, data is grouped by entity columns (e.g., `driver_id`) and aggregated before being passed to the transformation function.
45+
46+
**Important**: Aggregations and transformations are mutually exclusive. When aggregations are specified, they replace the transformation function.
47+
48+
### Usage
49+
50+
```python
51+
from feast import Aggregation
52+
from datetime import timedelta
53+
54+
@on_demand_feature_view(
55+
sources=[driver_hourly_stats_view],
56+
schema=[
57+
Field(name="total_trips", dtype=Int64),
58+
Field(name="avg_rating", dtype=Float64),
59+
],
60+
aggregations=[
61+
Aggregation(column="trips", function="sum"),
62+
Aggregation(column="rating", function="mean"),
63+
],
64+
)
65+
def driver_aggregated_stats(inputs):
66+
# No transformation function needed when using aggregations
67+
pass
68+
```
69+
70+
Aggregated columns are automatically named using the pattern `{function}_{column}` (e.g., `sum_trips`, `mean_rating`).
71+
4272
## Example
4373
See [https://github.com/feast-dev/on-demand-feature-views-demo](https://github.com/feast-dev/on-demand-feature-views-demo) for an example on how to use on demand feature views.
4474

‎sdk/python/docs/source/feast.infra.compute_engines.local.backends.rst‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,39 +7,39 @@ Submodules
77
feast.infra.compute\_engines.local.backends.base module
88
-------------------------------------------------------
99

10-
.. automodule:: feast.infra.compute_engines.local.backends.base
10+
.. automodule:: feast.infra.compute_engines.backends.base
1111
:members:
1212
:undoc-members:
1313
:show-inheritance:
1414

1515
feast.infra.compute\_engines.local.backends.factory module
1616
----------------------------------------------------------
1717

18-
.. automodule:: feast.infra.compute_engines.local.backends.factory
18+
.. automodule:: feast.infra.compute_engines.backends.factory
1919
:members:
2020
:undoc-members:
2121
:show-inheritance:
2222

2323
feast.infra.compute\_engines.local.backends.pandas\_backend module
2424
------------------------------------------------------------------
2525

26-
.. automodule:: feast.infra.compute_engines.local.backends.pandas_backend
26+
.. automodule:: feast.infra.compute_engines.backends.pandas_backend
2727
:members:
2828
:undoc-members:
2929
:show-inheritance:
3030

3131
feast.infra.compute\_engines.local.backends.polars\_backend module
3232
------------------------------------------------------------------
3333

34-
.. automodule:: feast.infra.compute_engines.local.backends.polars_backend
34+
.. automodule:: feast.infra.compute_engines.backends.polars_backend
3535
:members:
3636
:undoc-members:
3737
:show-inheritance:
3838

3939
Module contents
4040
---------------
4141

42-
.. automodule:: feast.infra.compute_engines.local.backends
42+
.. automodule:: feast.infra.compute_engines.backends
4343
:members:
4444
:undoc-members:
4545
:show-inheritance:

‎sdk/python/docs/source/feast.infra.compute_engines.local.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ Subpackages
77
.. toctree::
88
:maxdepth: 4
99

10-
feast.infra.compute_engines.local.backends
10+
feast.infra.compute_engines.backends
1111

1212
Submodules
1313
----------

sdk/python/feast/infra/compute_engines/local/backends/__init__.py renamed to sdk/python/feast/infra/compute_engines/backends/__init__.py

File renamed without changes.

sdk/python/feast/infra/compute_engines/local/backends/base.py renamed to sdk/python/feast/infra/compute_engines/backends/base.py

File renamed without changes.

sdk/python/feast/infra/compute_engines/local/backends/factory.py renamed to sdk/python/feast/infra/compute_engines/backends/factory.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
import pandas as pd
44
import pyarrow
55

6-
from feast.infra.compute_engines.local.backends.base import DataFrameBackend
7-
from feast.infra.compute_engines.local.backends.pandas_backend import PandasBackend
6+
from feast.infra.compute_engines.backends.base import DataFrameBackend
7+
from feast.infra.compute_engines.backends.pandas_backend import PandasBackend
88

99

1010
class BackendFactory:
@@ -46,7 +46,7 @@ def _is_polars(entity_df) -> bool:
4646

4747
@staticmethod
4848
def _get_polars_backend():
49-
from feast.infra.compute_engines.local.backends.polars_backend import (
49+
from feast.infra.compute_engines.backends.polars_backend import (
5050
PolarsBackend,
5151
)
5252

sdk/python/feast/infra/compute_engines/local/backends/pandas_backend.py renamed to sdk/python/feast/infra/compute_engines/backends/pandas_backend.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
import pandas as pd
44
import pyarrow as pa
55

6-
from feast.infra.compute_engines.local.backends.base import DataFrameBackend
6+
from feast.infra.compute_engines.backends.base import DataFrameBackend
77

88

99
class PandasBackend(DataFrameBackend):

sdk/python/feast/infra/compute_engines/local/backends/polars_backend.py renamed to sdk/python/feast/infra/compute_engines/backends/polars_backend.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
import polars as pl
44
import pyarrow as pa
55

6-
from feast.infra.compute_engines.local.backends.base import DataFrameBackend
6+
from feast.infra.compute_engines.backends.base import DataFrameBackend
77

88

99
class PolarsBackend(DataFrameBackend):

‎sdk/python/feast/infra/compute_engines/local/compute.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,10 @@
1212
MaterializationTask,
1313
)
1414
from feast.infra.common.retrieval_task import HistoricalRetrievalTask
15+
from feast.infra.compute_engines.backends.base import DataFrameBackend
16+
from feast.infra.compute_engines.backends.factory import BackendFactory
1517
from feast.infra.compute_engines.base import ComputeEngine
1618
from feast.infra.compute_engines.dag.context import ExecutionContext
17-
from feast.infra.compute_engines.local.backends.base import DataFrameBackend
18-
from feast.infra.compute_engines.local.backends.factory import BackendFactory
1919
from feast.infra.compute_engines.local.feature_builder import LocalFeatureBuilder
2020
from feast.infra.compute_engines.local.job import (
2121
LocalMaterializationJob,

‎sdk/python/feast/infra/compute_engines/local/feature_builder.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22

33
from feast.infra.common.materialization_job import MaterializationTask
44
from feast.infra.common.retrieval_task import HistoricalRetrievalTask
5+
from feast.infra.compute_engines.backends.base import DataFrameBackend
56
from feast.infra.compute_engines.feature_builder import FeatureBuilder
6-
from feast.infra.compute_engines.local.backends.base import DataFrameBackend
77
from feast.infra.compute_engines.local.nodes import (
88
LocalAggregationNode,
99
LocalDedupNode,

0 commit comments

Comments
 (0)