|
| 1 | +# Chronon offline store (contrib) |
| 2 | + |
| 3 | +## Description |
| 4 | + |
| 5 | +The Chronon offline store provides support for reading [ChrononSources](../data-sources/chronon.md) from Chronon's Parquet materialization output. |
| 6 | + |
| 7 | +Chronon remains the system of record for feature computation and materialization. Feast reads Chronon-produced data for historical retrieval, feature reuse, and registry-driven training workflows. |
| 8 | + |
| 9 | +## Getting started |
| 10 | + |
| 11 | +Chronon-backed feature repos use the Chronon offline store with Chronon sources: |
| 12 | + |
| 13 | +{% code title="feature_store.yaml" %} |
| 14 | +```yaml |
| 15 | +project: my_project |
| 16 | +registry: data/registry.db |
| 17 | +provider: chronon |
| 18 | +offline_store: |
| 19 | + type: chronon |
| 20 | +online_store: |
| 21 | + type: chronon |
| 22 | + path: http://localhost:8080 |
| 23 | +``` |
| 24 | +{% endcode %} |
| 25 | +
|
| 26 | +Example feature view: |
| 27 | +
|
| 28 | +```python |
| 29 | +from feast import Entity, FeatureView, Field |
| 30 | +from feast.infra.offline_stores.contrib.chronon_offline_store.chronon_source import ( |
| 31 | + ChrononSource, |
| 32 | +) |
| 33 | +from feast.types import Float32 |
| 34 | + |
| 35 | +driver = Entity(name="driver", join_keys=["driver_id"]) |
| 36 | + |
| 37 | +driver_stats = FeatureView( |
| 38 | + name="driver_stats", |
| 39 | + entities=[driver], |
| 40 | + schema=[Field(name="rating", dtype=Float32)], |
| 41 | + source=ChrononSource( |
| 42 | + name="driver_stats_source", |
| 43 | + materialization_path="data/chronon/driver_stats", |
| 44 | + chronon_join="team/driver_stats.v1", |
| 45 | + timestamp_field="event_timestamp", |
| 46 | + created_timestamp_column="created_timestamp", |
| 47 | + ), |
| 48 | +) |
| 49 | +``` |
| 50 | + |
| 51 | +## Historical retrieval |
| 52 | + |
| 53 | +`get_historical_features` performs point-in-time joins against Chronon's materialized Parquet data. For each entity row, Feast selects the latest Chronon row with an event timestamp at or before the entity timestamp. If `created_timestamp_column` is configured and duplicate event timestamps exist, the latest created row wins. |
| 54 | + |
| 55 | +Reads select only the required feature, entity-key, event-time, and (when needed) created-time columns from Parquet, translating field mappings to physical column names before loading. Joins still execute locally in pandas; column selection reduces I/O and memory use but does not provide distributed execution. |
| 56 | + |
| 57 | +Python on-demand feature views can transform the retrieved features locally through Feast's standard retrieval job. Put required request-time inputs in the pandas `entity_df`. `to_df()`, `to_arrow()`, and saved datasets include the requested transformed outputs. Time-range retrieval without an `entity_df` cannot supply request-time inputs. |
| 58 | + |
| 59 | +This is intended for training and validation workflows that want Feast's registry and retrieval APIs while using Chronon as the feature computation engine. |
| 60 | + |
| 61 | +## Save retrieval results |
| 62 | + |
| 63 | +Use `provider: chronon` and `SavedDatasetFileStorage` to persist a retrieval result as Parquet and load it again through Feast: |
| 64 | + |
| 65 | +```python |
| 66 | +from feast.infra.offline_stores.file_source import SavedDatasetFileStorage |
| 67 | + |
| 68 | +job = store.get_historical_features( |
| 69 | + entity_df=entity_df, |
| 70 | + features=["driver_stats:rating"], |
| 71 | +) |
| 72 | +training = store.create_saved_dataset( |
| 73 | + from_=job, |
| 74 | + name="training_v1", |
| 75 | + storage=SavedDatasetFileStorage(path="data/training_v1.parquet"), |
| 76 | +) |
| 77 | +training_df = store.get_saved_dataset("training_v1").to_df() |
| 78 | +``` |
| 79 | + |
| 80 | +Relative paths resolve against the feature repository. File storage can also use a Parquet directory or supported PyArrow filesystem URI, including S3 with the existing file-storage credentials/endpoint configuration. Existing destinations are rejected unless `allow_overwrite=True`; overwriting a directory replaces its contents. Use a dedicated output location. Writes are synchronous and are not an atomic publication mechanism. |
| 81 | + |
| 82 | +The saved result retains request columns, custom entity timestamp names, full feature names, and already-computed on-demand features. This saves training results; it does not write feature values into Chronon's computation or online storage. |
| 83 | + |
| 84 | +## Configuration reference |
| 85 | + |
| 86 | +| Parameter | Required | Default | Description | |
| 87 | +| :-------- | :------- | :------ | :---------- | |
| 88 | +| `type` | yes | - | Must be set to `chronon`. | |
| 89 | +| `path` | no | - | Reserved for future offline store configuration. Source-level `materialization_path` controls where data is read from. | |
| 90 | + |
| 91 | +## Functionality Matrix |
| 92 | + |
| 93 | +The set of functionality supported by offline stores is described in detail [here](overview.md#functionality). |
| 94 | +Below is a matrix indicating which functionality is supported by the Chronon offline store. |
| 95 | + |
| 96 | +| | Chronon | |
| 97 | +| :----------------------------------------------------------------- | :------ | |
| 98 | +| `get_historical_features` (point-in-time correct join) | yes | |
| 99 | +| `pull_latest_from_table_or_query` (retrieve latest feature values) | yes | |
| 100 | +| `pull_all_from_table_or_query` (retrieve a saved dataset) | yes | |
| 101 | +| `offline_write_batch` (persist dataframes to offline store) | no | |
| 102 | +| `write_logged_features` (persist logged features to offline store) | no | |
| 103 | + |
| 104 | +Below is a matrix indicating which functionality is supported by `ChrononRetrievalJob`. |
| 105 | + |
| 106 | +| | Chronon | |
| 107 | +| ----------------------------------------------------- | ------- | |
| 108 | +| export to dataframe | yes | |
| 109 | +| export to arrow table | yes | |
| 110 | +| export to arrow batches | no | |
| 111 | +| export to SQL | no | |
| 112 | +| export to data lake (S3, GCS, etc.) | no | |
| 113 | +| export to data warehouse | no | |
| 114 | +| export as Spark dataframe | no | |
| 115 | +| local execution of Python-based on-demand transforms | yes | |
| 116 | +| remote execution of Python-based on-demand transforms | no | |
| 117 | +| persist results in the offline store | yes (Parquet saved datasets) | |
| 118 | +| preview the query plan before execution | no | |
| 119 | +| read partitioned data | yes | |
| 120 | + |
| 121 | +To compare this set of functionality against other offline stores, please see the full [functionality matrix](overview.md#functionality-matrix). |
0 commit comments