Skip to content

Inconsistent behavior between File and Snowflake offline stores #2960

Closed

Description

@cburroughs

Expected Behavior

With equivalent definitions and the same data in each offline store, get_historical_features should return the same results regardless of backing store.

Current Behavior

With the same underlying data, equivalent definitions and given the same entity data frame, File and Snowflake offline stores return different results.

$ python clear.py  query-file
      event_timestamp node
0 2015-10-21 02:00:00  foo
1 2015-10-21 03:00:00  foo
result:
            event_timestamp node  price  temp_f
0 2015-10-21 02:00:00+00:00  foo      6      67
1 2015-10-21 03:00:00+00:00  foo      6      67
$ python clear.py  query-snow
      event_timestamp node
0 2015-10-21 02:00:00  foo
1 2015-10-21 03:00:00  foo
result:
      event_timestamp node  price  temp_f
0 2015-10-21 02:00:00  foo      5      50
1 2015-10-21 03:00:00  foo      6      67

Steps to reproduce

# download files; adjust credentials and db names
$ feast apply
$ python clear.py write-dummy-data-snowflake
$ python clear.py query-snow
# Switch which section is commented out in yaml
$ python clear.py write-dummy-data-file
$ python clear.py query-file

Specifications

  • Version: Feast SDK Version: "feast 0.22.1"
  • Platform x86_64 on Python 3.9.12
  • Subsystem: Linux 5.4.188

Possible Solution

I presume it is incorrect that these are returning different results, but if this is undefined behavior where both are in some sense legitament, which behavior in Feast is undefined should be more explicit in the docs.

Activity

  1. felixwang9817 commented on Jul 26, 2022

    @felixwang9817
    Collaborator

    hey @cburroughs thanks for reporting this! by inspection it's clear the file offline store is wrong and Snowflake is correct

    the root source of the issue was that the file offline store had some incorrect logic for handling feature views without ttls; #2971 should solve it! (adding ttls to the feature views defined also solves the problem)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions