Allow On Demand Feature Views w/ RequestSource Only

Is your feature request related to a problem? Please describe.

No

Currently there's a requirement that an on-demand feature view to be associated w/ a FeatureView source, but there's a scenario where the user just provides a RequestSource. The example provided demonstrates getting an entity_df, passing it to a FeatureView and then using the result of that FeatureView to return additional, on-demand feature values.

entity = Entity(
    name="entity",
    description="Entity",
    join_keys=["entity_id"],
    tags={},
    value_type=ValueType.INT64,
)

input_request = RequestSource(
    name="input_request",
    schema=[
        Field(name="datetime", dtype=Float32),
        Field(name="value", dtype=Float32),
    ],
)

def calc_max_value(input: pd.DataFrame) -> pd.DataFrame:
    result = pd.DataFrame(index=input.index)

    # plan phase hack
    if region.join_key not in input.columns:
        result["max_value"] = 0.0
        return result

    result["max_value"] = input.groupby(
        [input[entity.join_key], input["datetime"].dt.date]
    )["value"].transform("max")

    return result


max_value_odfv = on_demand_feature_view(
    name="max_value",
    entities=[entity],
    sources=[input_request],  # ONLY a RequestSource datasource
    schema=[
        Field(name="max_value", dtype=Float64),
    ],
    mode="pandas",
)(calc_max_value)

Then we can chain the output of a FeatureView to the input of an On-Demand Feature View like:

entity_sql = "SELECT ..."

value_df = store.get_historical_features(
    entity_df=entity_sql,
    features=[
        'feature:value'
    ]
)

max_value_df = store.get_historical_features(
    entity_df=value_df,
    features=[
        'max_value:max_value'
    ]
)

Describe the solution you'd like
I would like to be able to provide user defined functions that allow additional transformations on dataframes, not directly tied to a FeatureView.

Describe alternatives you've considered
None have been considered at this point.

Additional context
I hacked around w/ the postgres implementation and simply just modifying the query(dropping a comma) to assume no featureviews:

{% if featureviews | length > 0 %}
,
{% endif %}

{% for featureview in featureviews %}

Is this not available today?

I have this test:

feast/sdk/python/tests/unit/test_on_demand_python_transformation.py

Line 1121 in 4b91f26

@on_demand_feature_view(

Which does it.

Hmm. One difference I notice right away is the mode. You're using python and I'm using pandas.

Should we be using mode=python for batch transformations? I was using pandas because I assumed it was preferred for larger data sets based on the docs - https://docs.feast.dev/master/reference/beta-on-demand-feature-view#transformation-modes

@franciscojavierarceo

@franciscojavierarceo I wonder if this is just a bug in the postgres offline store implementation of get_historical_features? I can file a bug if you think so. The query will definitely break though if you look here:

entity_dataframe AS (
    SELECT *,
        {{entity_df_event_timestamp_col}} AS entity_timestamp
        {% for featureview in featureviews %}
            {% if featureview.entities %}
            ,(
                {% for entity in featureview.entities %}
                    CAST("{{entity}}" as VARCHAR) ||
                {% endfor %}
                CAST("{{entity_df_event_timestamp_col}}" AS VARCHAR)
            ) AS "{{featureview.name}}__entity_row_unique_id"
            {% else %}
            ,CAST("{{entity_df_event_timestamp_col}}" AS VARCHAR) AS "{{featureview.name}}__entity_row_unique_id"
            {% endif %}
        {% endfor %}
    FROM
        {% if entity_select_mode == "embed_query" %}
            entity_query
        {% else %}
            {{ left_table_query_string }}
        {% endif %}
),

{% for featureview in featureviews %}

...

SELECT "{{ final_output_feature_names | join('", "')}}"
FROM entity_dataframe
{% for featureview in featureviews %}
LEFT JOIN (
    SELECT
        "{{featureview.name}}__entity_row_unique_id"
        {% for feature in featureview.features %}
            ,"{% if full_feature_names %}{{ featureview.name }}__{{featureview.field_mapping.get(feature, feature)}}{% else %}{{ featureview.field_mapping.get(feature, feature) }}{% endif %}"
        {% endfor %}
    FROM "{{ featureview.name }}__cleaned"
) AS "{{featureview.name}}" USING ("{{featureview.name}}__entity_row_unique_id")
{% endfor %}

The above shows that trailing comma that will break the query if it's not dropped before SELECT is called.

I also believe this will be an issue w/ RedShift.

@blaketastic2

this is done right @blaketastic2 ?

I can only confirm for the Postgres offline store. I believe the RedShift impl will have the same issue.

Navigation Menu

Repository navigation

Allow On Demand Feature Views w/ RequestSource Only #5239

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Allow On Demand Feature Views w/ RequestSource Only #5239

Description

Activity

franciscojavierarceo commented on Apr 8, 2025

blaketastic2 commented on Apr 8, 2025

blaketastic2 commented on Apr 8, 2025

blaketastic2 commented on Apr 9, 2025

franciscojavierarceo commented on May 12, 2025

blaketastic2 commented on May 12, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions