Repository navigation
Description
Expected Behavior
With no schema on the feature view, feast apply infers a Field for each source column with the column's type (docs, feature-view.md § Feature inferencing). The DataFrame inference path already maps the same values: datetime.date → UnixTimestamp, category → String, Decimal → Decimal. A Parquet file written by pandas with those columns should infer the same way, or fail with a message that names the column and the unsupported type.
Current Behavior
feast apply fails with a bare KeyError carrying the Arrow type string, as soon as the Parquet file has one such column, even when that column is not wanted as a feature.
Column type as pandas writes it and pyarrow.parquet.ParquetDataset(path).schema reads it back, then schema-less apply of a feature view on a file holding only driver_id, event_timestamp and that column:
| pandas column | Arrow type read | schema-less apply |
|---|---|---|
| int64, float64, str, bytes, list of int (controls) | int64, double, string, binary, list<element: int64> |
applied |
datetime.date values |
date32[day] |
KeyError: 'date32[day]' |
category dtype |
dictionary<values=string, indices=int8, ordered=0> |
KeyError |
Decimal values |
decimal128(3, 2) |
KeyError |
The same file applies when the feature view lists its features explicitly, so that is the workaround. decimal256, time32, time64, duration, large_binary and halffloat columns fail the same way. Same result on 0.67.0 and on current master.
Steps to reproduce
feature_store.yaml:
project: cli_repro
registry: data/registry.db
provider: local
online_store:
type: sqlite
path: data/online.db
entity_key_serialization_version: 3make_data.py:
from datetime import date, datetime, timedelta, timezone
import pandas as pd
now = datetime.now(timezone.utc).replace(microsecond=0)
pd.DataFrame({
"driver_id": [1, 2],
"event_timestamp": [now - timedelta(hours=1), now - timedelta(hours=2)],
"trips": [10, 20],
"signup_date": [date(2026, 1, 1), date(2026, 1, 2)],
}).to_parquet("data/driver_stats.parquet")definitions.py:
import os
from datetime import timedelta
from feast import Entity, FeatureView, Field, FileSource
from feast.types import Int64
driver = Entity(name="driver", join_keys=["driver_id"])
source = FileSource(path="data/driver_stats.parquet", timestamp_field="event_timestamp")
schema = [Field(name="trips", dtype=Int64)] if os.environ.get("SCHEMA") == "explicit" else None
driver_stats = FeatureView(name="driver_stats", entities=[driver], ttl=timedelta(days=1), schema=schema, source=source)mkdir -p data && python make_data.py
feast apply # exit 1
SCHEMA=explicit feast apply # exit 0, feature trips: Int64Tail of the failing run:
File ".../feast/inference.py", line 269, in _infer_features_and_entities
fv.batch_source.source_datatype_to_feast_value_type()(
File ".../feast/type_map.py", line 1661, in pa_to_feast_value_type
value_type = type_map[pa_type_as_str]
KeyError: 'date32[day]'
Replacing signup_date with pd.Series(["x", "y"], dtype="category") gives KeyError: 'dictionary<values=string, indices=int8, ordered=0>'; with [Decimal("1.10"), Decimal("2.20")] it gives KeyError: 'decimal128(3, 2)'.
Specifications
- Version: 0.67.0; same on current master
- Platform: macOS arm64, Python 3.11, pandas 2.3.3, pyarrow 26.0.0
- Subsystem: Python SDK, feature inference for
FileSource(Parquet) duringfeast apply
Possible Solution
pa_to_feast_value_type in sdk/python/feast/type_map.py looks the type string up in a fixed dict after the timestamp, map, large_string and struct branches, so any other spelling raises. Mapping date32/date64 to UNIX_TIMESTAMP, decimal128/decimal256 to DECIMAL and dictionary<values=T, ...> to the mapping of T would match what the DataFrame path does; failing that, a ValueError naming the type would at least point at the column. I searched issues and PRs for the symptom and the mechanism and found only #6876, which handles the list<element: T> and large_list spellings and does not touch scalar types. One related question: a polars-written string column arrives as large_string and is registered as Json rather than String; is that intended? Happy to open a PR for the mapping.
Activity
Mrun25 commentedon Oct 19, 2026
Verified on current main (7903e2e): schema-less
feast applyover a ParquetFileSourcecrashes inpa_to_feast_value_type(sdk/python/feast/type_map.py), which looks the Arrow type string up in a fixed dict -date32[day],dictionary<...>(pandas category) anddecimal128(...)all miss, raising a bareKeyError. The DataFrame inference path already maps these (date-> UnixTimestamp,category-> String,decimal-> Decimal). Plan: add the missing Arrow mappings to match the DataFrame path, and replace the bareKeyErrorwith a descriptive inference error that names the column and the unsupported type, with unit tests. I will open a PR shortly.Mrun25 commentedon Oct 19, 2026
Opened a fix: #7043
fix: infer Feast types for additional Arrow types and report column n…
fix: infer Feast types for additional Arrow types and report column n…