Feature set versioning #386

Activity

1. woop commented on Jan 4, 2020

   

   woop

   on Jan 4, 2020

   Last edited by woop

   Member

   More actions

   One of the motivations for versions was to allow for a Feast Serving deployment to subscribe to specific feature sets which dont change over time. It would then decouple the users that modify feature sets from production systems.

   The fear was that if we didnt have versions then users would implement their own versioning strategies in the feature set name itself, which would result in loads of feature sets in the global namespace.

   My thoughts on the current state of Feast as it relates to versions:

   • Given that we have implemented project level isolation, I believe that versions are even less useful than before, and in fact they simply add unnecessary complexity.
   • Data scientists have argued that versions dont add any value to their workflow.
   • @thirteen37 has argued that in most database systems that it is up to the administrator to ensure safe migrations from one state to another. For example if there is a production dependency on a table then the administrator should ensure that the correct columns are added, data is backfilled, that both the previous and future clients are supported, and that a roll back can happen if something goes wrong. Taking on this responsibility within Feast adds a lot of complexity that seems outside of our capabilities and/or value proposition.

   To get back to your proposals above, which I will number (1) and (2).

   1. This is already implemented. Users do not have to provide versions when requesting features for retrieval. However, this is only happening on the retrieval side, not ingestion. It also does not address all issues with versions and the complexity they introduce.
   2. I am more inclined to take this approach. We can scrap versions altogether.
2. woop commented on Jan 4, 2020

   

   woop

   on Jan 4, 2020

   Last edited by woop

   Member

   More actions

   Below is a straw-man proposal for us to attack. Let's assume that we remove versions entirely. The functionality is then as follows

   Terminology:

   • Ingestion: This includes the Python SDK that writes to Kafka or a stream that pushes feature rows to Kafka.
   • Population: Jobs that consume feature rows from a Kafka and populates a Feast store like Redis, BigQuery, Hive, or Cassandra.

   Functionality:

   • Users can still create, register, and manage feature sets through the Core API as well as SDK.
   • Users can add new features. This adds a new column in an RDBMS like BigQuery or extra fields when writing to a KV store like Redis.
   • Users can delete features. This does not remove columns in an RDBMS, but simply marks them as deprecated. These columns/fields cannot be reused and cannot be queried. They are invisible to end users, but the data itself continues to exist in stores unless garbage collection is implemented.
   • Users can not update a feature type of an existing feature. This can possibly be implemented at a later date if it is necessary, but it does not seem to be worth the added development work.
   • Users can not add, remove, or modify entities in their feature set. It's not clear that there is a strong need for this, and it's also not clear how this will impact certain stores that require indexing on entities.

   Implementation:

   • Versions are not exposed to end users. Not at the feature set level and especially not at feature retrieval. Optionally we can still maintain version tracking as a form of metadata, but it would only be a counter that increments upon change to a feature set. It has no explicit functionality in our APIs.
   • Ingestion does not depend on versions. Users can do either a batch ingestion or streaming ingestion to a Kafka topic based purely on a feature set name, as well as the fields within a feature set at that point in time.
   • The population jobs that fill up stores don't care about versions. They parse source data on the stream based on their current understanding of feature sets and feature row contents.
     • If there are missing or erroneous fields in a feature row then a failure occurs. Rows are sent to a deadletter queue. "deleted" features are allowed to be missing.
     • If there are additional/unwanted fields in a feature row then these are ignored and the rest of the row is parsed. This ties in to Filter out extra fields, deduplicate fields in ingestion #404
     • When a population job writes to a store, it writes null values to any deleted feature/field.
   • If a change is made to a feature set then the feature set moves to an updating state. Batch ingestions cannot be started until these changes are applied. Once the changes are applied, batch ingestion can function again and will use the latest schema. Any ingestion that has already been running will still continue to run, including streaming ingestion.
   • Retrieval will allow for additional columns or fields to exist, but will simply ignore them. If fields are missing, then retrieval should fail. It's possible that different "versions" of a feature set could be looked up in a single query, for example multiple keys for an online retrieval could have different fields. As long as the correct fields exist for the feature set at that point in time, the query should succeed.
3. woop

   added

   kind/discussion

   kind/featureNew feature or requestNew feature or request

   priority/p1

   and removed

   kind/discussion

   on Jan 26, 2020
4. woop

   added this to the v0.5.0 milestone on Mar 10, 2020
5. woop

   added

   kind/housekeeping

   kind/techdebt

   on Mar 10, 2020
6. zhilingc commented on Mar 18, 2020

   

   zhilingc

   on Mar 18, 2020

   Collaborator

   More actions

   I've drafted a quick RFC for this, in hopes of picking this up soon (hopefully after the storage refactor).

   https://docs.google.com/document/d/1P44LHd724JloQtpn10naAg5MlzcrkeOlN3QzAn9O8uY

   ❤️React with ❤️2Reacted by Ches Martin and Kshitiz Lohia
7. ches

   pinned this issue on Mar 28, 2020
8. feast-ci-bot

   closed this as completedin #676on May 13, 2020
9. woop

   unpinned this issue on May 24, 2020