Repository navigation
Conversation
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
b2cc986 to
2f38847
Compare
luc-pimentel
left a comment
luc-pimentel
left a comment
There was a problem hiding this comment.
Thanks for picking up the issue I opened. I ran its reproduction against this PR's current head: the Arrow type strings as FileSource reads them from a pandas-written Parquet file, a schema-less feast apply on a file holding one such column, and the CLI repro from the issue.
| pandas column | Arrow type read | master | this PR |
|---|---|---|---|
datetime.date values |
date32[day] |
KeyError |
applied as UnixTimestamp |
category dtype |
dictionary<values=string, indices=int8, ordered=0> |
KeyError |
applied as String |
Decimal values |
decimal128(3, 2) |
KeyError |
applied as Decimal |
| int64, float64, str, bytes, list of int (controls) | int64, double, string, binary, list<element: int64> |
applied | applied, same types |
The issue's feast apply repro exits 0 here and registers trips: Int64 and signup_date: UnixTimestamp; on master it exits 1 with KeyError: 'date32[day]'. decimal256, large_binary and halffloat columns map as well. time32, time64, duration and a polars-written large_list column now fail with the new error naming the column and the Arrow type instead of the bare KeyError, which is what the issue asked for as the fallback. The DataFrame inference path gives the same types as before.
Tested with Python 3.11, pandas 2.3.3, pyarrow 25.0.1 and polars 2.1.0 on 2026-10-26.
Looks good to me. Curious what the maintainers think.
Mrun25
changed the title
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #7043 +/- ##
=======================================
Coverage 52.45% 52.46%
=======================================
Files 459 459
Lines 58172 58192 +20
Branches 8577 8583 +6
=======================================
+ Hits 30515 30530 +15
- Misses 25573 25577 +4
- Partials 2084 2085 +1
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
…ame on inference failure (feast-dev#7040) Signed-off-by: Mrun25 <[email protected]>
54fcf16 to
f2829bd
Compare
Fixes #7040.
A Parquet file written by pandas fails as a batch source: pa_to_feast_value_type raises a bare KeyError for Arrow types pandas commonly writes (date32/date64, decimal128/256, dictionary-encoded categories, halffloat, large_binary), even though the DataFrame inference path maps the equivalent pandas dtypes fine.
Changes:
Tests: parametrized mapping tests and a descriptive-error test in tests/unit/test_type_map.py. ruff format and check clean.