Skip to content

fix: Validate entity column types in feast dbt import - #7067

Merged
shuchu merged 2 commits into
feast-dev:masterfrom
NopeCharon:fix/dbt-entity-column-type-warning
Nov 10, 2026
Merged

fix: Validate entity column types in feast dbt import#7067
shuchu merged 2 commits into
feast-dev:masterfrom
NopeCharon:fix/dbt-entity-column-type-warning

Conversation

@NopeCharon

@NopeCharon NopeCharon commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

feast dbt import only checks that the entity columns exist. Their type is never looked at:

  • A model whose entity column is declared FLOAT64, BOOLEAN, BYTES or TIMESTAMP imports silently, and Feast creates an entity keyed on a floating point, boolean, binary or timestamp value.
  • A model with an ARRAY<...> entity column crashes the whole import with an unhandled ValueError: Entity driver_id has type ValueType.STRING, which does not match the inferred type Array(...) raised from FeatureView.__init__.

With this PR, the import maps each entity column's dbt type through the existing map_dbt_type_to_feast_type mapping and then:

  • Array entity column: the model is skipped with a warning (Model '...' has array entity column(s): ... Skipping.). This matches how models missing an entity or timestamp column are handled.
  • Any type other than String, Int32 or Int64: a warning names the dbt type and the Feast type it maps to, and the model is still imported. For example: Warning: Model 'driver_features' entity column 'conv_rate' has type 'float64' (Feast type Float64), which may not be suitable as an entity key. Recommended types: STRING, INT64. Importing anyway. This also flags Snowflake's NUMBER(38,0), which the mapping turns into Float64.
  • No data_type: columns without one default to STRING and do not warn.

The models used for the summary and for --output code generation are now collected as they are processed, instead of being recomputed afterwards from column presence. Otherwise a skipped model would still be written into the generated file.

The dbt how-to documents the check and the two new warnings in its Requirements and Troubleshooting sections.

Which issue(s) this PR fixes:

Fixes #5876

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

Unit tests: the new sdk/python/tests/unit/dbt/test_entity_column_type.py builds manifests and runs feast dbt import through the CLI. It covers:

  • a warning for FLOAT64, NUMBER(38,0), BOOLEAN, BYTES and TIMESTAMP, with the model still imported
  • no warning for STRING, VARCHAR(36), INT64, INTEGER, NUMBER(10,0) and an undeclared type
  • with several entity columns, only the unsuitable one warns
  • a model with an ARRAY<INT64> entity column is skipped, and with --output it is left out of the generated file while the other model is kept

Seven of these cases fail on master (the array case with the ValueError above). All 50 tests in tests/unit/dbt pass.

Integration tests: make test-python-integration-dbt (dbt-core + dbt-duckdb) passes, 84 tests.

Manual tests: ran feast dbt import --dry-run against the integration project's generated manifest with -e driver_id (no warning) and -e conv_rate (warning shown).

Static checks: scoped ruff check, ruff format --check and mypy are clean.

@NopeCharon
fix: Validate entity column types in feast dbt import
e34f438 
`feast dbt import` only checked that the entity columns exist. A model
whose entity column is declared as FLOAT64, BOOLEAN, BYTES or TIMESTAMP
was imported silently, producing an entity keyed on a floating point,
boolean, binary or timestamp value. A model with an ARRAY entity column
crashed the whole import with an unhandled ValueError from FeatureView.

The import now maps each entity column's dbt type through the existing
dbt-to-Feast type mapping and:

- skips models with an array entity column, with a warning, in the same
  way as models that miss an entity or timestamp column;
- prints a warning for entity columns that map to anything other than
  String, Int32 or Int64, naming the dbt type and the Feast type it maps
  to, and still imports the model. This also flags Snowflake's
  NUMBER(38,0), which the mapping turns into Float64.

Columns without a declared data_type default to STRING and do not warn.

The list of imported models used for the summary and for --output code
generation is now collected while processing models instead of being
recomputed afterwards from column presence, so skipped models can no
longer leak into the generated file.

The dbt how-to documents the check and the new warnings.

Fixes feast-dev#5876

Signed-off-by: NopeCharon <[email protected]>
@NopeCharon
NopeCharon requested a review from a team as a code owner October 23, 2026 16:46
@codecov-commenter

codecov-commenter commented Nov 1, 2026 •
edited
Loading

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 51.72%. Comparing base (bbaca76) to head (b5e1dc7).
⚠️ Report is 11 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7067      +/-   ##
==========================================
+ Coverage   51.32%   51.72%   +0.40%     
==========================================
  Files         458      458              
  Lines       57441    57545     +104     
  Branches     8421     8445      +24     
==========================================
+ Hits        29482    29768     +286     
+ Misses      25950    25752     -198     
- Partials     2009     2025      +16
Flag Coverage Δ *Carryforward flag
go-feature-server 30.58% <ø> (ø) Carriedforward from e34f438
python-unit 53.22% <100.00%> (+0.43%) ⬆️

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
sdk/python/feast/cli/dbt_import.py 54.77% <100.00%> (+38.33%) ⬆️

... and 9 files with indirect coverage changes

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b1f2168...b5e1dc7. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

shuchu
shuchu approved these changes Nov 10, 2026
View reviewed changes

@shuchu shuchu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@shuchu
shuchu merged commit abca80c into feast-dev:master Nov 10, 2026
22 of 25 checks passed
@patelchaitany patelchaitany mentioned this pull request Nov 10, 2026
Merged
This was referenced Nov 10, 2026
Open
Open
franciscojavierarceo pushed a commit that referenced this pull request Nov 10, 2026
@feast-ci-bot
chore(release): release 0.68.0
fb5e5d5 
# [0.68.0](v0.67.0...v0.68.0) (2026-11-10)

### Bug Fixes

* add watch verb on clusterroles and clusterrolebindings for feast-operator ([#7062](#7062)) ([b6bdda4](b6bdda4))
* Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([#6993](#6993)) ([f416309](f416309))
* Break event timestamp ties by created timestamp in Milvus write batches ([#7084](#7084)) ([bbaca76](bbaca76))
* Break event timestamp ties by created timestamp in the Dask offline store ([#6952](#6952)) ([eb93b08](eb93b08))
* Build the Go feature server image in two stages ([1e5e90c](1e5e90c)), closes [#7060](#7060)
* Fall back to the serialized body when UDF source misses globals ([#6816](#6816)) ([f61079f](f61079f))
* Install patched libbrotli1 to address DSA-6582-1 ([#7096](#7096)) ([d458393](d458393))
* Keep description, tags and nested types of ODFV schema fields ([8ef6e75](8ef6e75))
* Keep entity columns out of StreamFeatureView proto features ([#7033](#7033)) ([d520584](d520584))
* Keep the metrics server running when a Gunicorn worker exits ([#7012](#7012)) ([68c5d54](68c5d54)), closes [#7011](#7011)
* Key registry proto caches on the proto, not its id() ([#7085](#7085)) ([147adad](147adad))
* Log ODFV materialization skips instead of printing them ([#7117](#7117)) ([8fdcee7](8fdcee7))
* Pin build dependencies of source builds to the CI lock ([#7104](#7104)) ([dec2f24](dec2f24))
* Populate spec.project in registry REST feature view responses (#… ([#7037](#7037)) ([e304a84](e304a84))
* Preserve colliding source timestamps in Dask retrieval ([#7071](#7071)) ([1396c88](1396c88))
* Preserve Dask historical requests without eligible features ([#7000](#7000)) ([fff09f7](fff09f7))
* Read None elements of UUID and Decimal lists as None ([#7039](#7039)) ([03f30fe](03f30fe))
* Retire the support of Python3.10 ([#7101](#7101)) ([a56766e](a56766e)), closes [#7104](#7104) [#7089](#7089) [#7106](#7106) [#7105](#7105)
* Serialize UUID and TimeUuid entity keys ([#7032](#7032)) ([0c845a1](0c845a1))
* Stop the Ray engine from hiding materialization errors ([#7026](#7026)) ([7903e2e](7903e2e)), closes [#7024](#7024)
* **ui:** stop treating the monitoring baseline as the latest window ([#7056](#7056)) ([0f01802](0f01802)), closes [#7054](#7054)
* Use date-only bounds for partition filters built from string timestamps ([#7116](#7116)) ([b1f2168](b1f2168))
* Validate entity column types in feast dbt import ([#7067](#7067)) ([abca80c](abca80c)), closes [#5876](#5876)
* Warm up the whole DynamoDB connection pool ([#6985](#6985)) ([8ce548c](8ce548c))

### Features

* Add --gc-freeze option to feast serve ([#7089](#7089)) ([aad0c15](aad0c15))
* Add native complex types and scalar index tags to the Milvus online store ([#7120](#7120)) ([f7068d6](f7068d6))
* Add num_partitions and partition_key_isolation to the Milvus online store ([#7083](#7083)) ([68de84e](68de84e)), closes [#6940](#6940)
* Add Pinecone online store integration ([#6585](#6585)) ([7a84f2a](7a84f2a))
* Add PolarisSource and PolarisProvider for Apache Polaris ([#7006](#7006)) ([386759c](386759c))
* Add row deletion to the Milvus online store ([#7082](#7082)) ([d3e2041](d3e2041))
* Add upstream FeatureView lineage support for PushSource ([#6853](#6853)) ([c9b9d72](c9b9d72))
* Optional nameless pre-computed vectors with a schema fingerprint ([#7077](#7077)) ([9420fac](9420fac))

### Performance Improvements

* Scan Lance sources lazily in the DuckDB offline store ([1299e4e](1299e4e))
@patelchaitany patelchaitany mentioned this pull request Nov 11, 2026
Merged
shuchu added a commit to shuchu/feast that referenced this pull request Nov 11, 2026
@NopeCharon @shuchu
fix: Validate entity column types in feast dbt import (feast-dev#7067)
789584c 
`feast dbt import` only checked that the entity columns exist. A model
whose entity column is declared as FLOAT64, BOOLEAN, BYTES or TIMESTAMP
was imported silently, producing an entity keyed on a floating point,
boolean, binary or timestamp value. A model with an ARRAY entity column
crashed the whole import with an unhandled ValueError from FeatureView.

The import now maps each entity column's dbt type through the existing
dbt-to-Feast type mapping and:

- skips models with an array entity column, with a warning, in the same
  way as models that miss an entity or timestamp column;
- prints a warning for entity columns that map to anything other than
  String, Int32 or Int64, naming the dbt type and the Feast type it maps
  to, and still imports the model. This also flags Snowflake's
  NUMBER(38,0), which the mapping turns into Float64.

Columns without a declared data_type default to STRING and do not warn.

The list of imported models used for the summary and for --output code
generation is now collected while processing models instead of being
recomputed afterwards from column presence, so skipped models can no
longer leak into the generated file.

The dbt how-to documents the check and the new warnings.

Fixes feast-dev#5876

Signed-off-by: NopeCharon <[email protected]>
Co-authored-by: Shuchu Han <[email protected]>
Signed-off-by: Shuchu Han <[email protected]>
shuchu pushed a commit to shuchu/feast that referenced this pull request Nov 11, 2026
@feast-ci-bot @shuchu
chore(release): release 0.68.0
5b94c9e 
# [0.68.0](feast-dev/feast@v0.67.0...v0.68.0) (2026-11-10)

### Bug Fixes

* add watch verb on clusterroles and clusterrolebindings for feast-operator ([feast-dev#7062](feast-dev#7062)) ([b6bdda4](feast-dev@b6bdda4))
* Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([feast-dev#6993](feast-dev#6993)) ([f416309](feast-dev@f416309))
* Break event timestamp ties by created timestamp in Milvus write batches ([feast-dev#7084](feast-dev#7084)) ([bbaca76](feast-dev@bbaca76))
* Break event timestamp ties by created timestamp in the Dask offline store ([feast-dev#6952](feast-dev#6952)) ([eb93b08](feast-dev@eb93b08))
* Build the Go feature server image in two stages ([1e5e90c](feast-dev@1e5e90c)), closes [feast-dev#7060](feast-dev#7060)
* Fall back to the serialized body when UDF source misses globals ([feast-dev#6816](feast-dev#6816)) ([f61079f](feast-dev@f61079f))
* Install patched libbrotli1 to address DSA-6582-1 ([feast-dev#7096](feast-dev#7096)) ([d458393](feast-dev@d458393))
* Keep description, tags and nested types of ODFV schema fields ([8ef6e75](feast-dev@8ef6e75))
* Keep entity columns out of StreamFeatureView proto features ([feast-dev#7033](feast-dev#7033)) ([d520584](feast-dev@d520584))
* Keep the metrics server running when a Gunicorn worker exits ([feast-dev#7012](feast-dev#7012)) ([68c5d54](feast-dev@68c5d54)), closes [feast-dev#7011](feast-dev#7011)
* Key registry proto caches on the proto, not its id() ([feast-dev#7085](feast-dev#7085)) ([147adad](feast-dev@147adad))
* Log ODFV materialization skips instead of printing them ([feast-dev#7117](feast-dev#7117)) ([8fdcee7](feast-dev@8fdcee7))
* Pin build dependencies of source builds to the CI lock ([feast-dev#7104](feast-dev#7104)) ([dec2f24](feast-dev@dec2f24))
* Populate spec.project in registry REST feature view responses (#… ([feast-dev#7037](feast-dev#7037)) ([e304a84](feast-dev@e304a84))
* Preserve colliding source timestamps in Dask retrieval ([feast-dev#7071](feast-dev#7071)) ([1396c88](feast-dev@1396c88))
* Preserve Dask historical requests without eligible features ([feast-dev#7000](feast-dev#7000)) ([fff09f7](feast-dev@fff09f7))
* Read None elements of UUID and Decimal lists as None ([feast-dev#7039](feast-dev#7039)) ([03f30fe](feast-dev@03f30fe))
* Retire the support of Python3.10 ([feast-dev#7101](feast-dev#7101)) ([a56766e](feast-dev@a56766e)), closes [feast-dev#7104](feast-dev#7104) [feast-dev#7089](feast-dev#7089) [feast-dev#7106](feast-dev#7106) [feast-dev#7105](feast-dev#7105)
* Serialize UUID and TimeUuid entity keys ([feast-dev#7032](feast-dev#7032)) ([0c845a1](feast-dev@0c845a1))
* Stop the Ray engine from hiding materialization errors ([feast-dev#7026](feast-dev#7026)) ([7903e2e](feast-dev@7903e2e)), closes [feast-dev#7024](feast-dev#7024)
* **ui:** stop treating the monitoring baseline as the latest window ([feast-dev#7056](feast-dev#7056)) ([0f01802](feast-dev@0f01802)), closes [feast-dev#7054](feast-dev#7054)
* Use date-only bounds for partition filters built from string timestamps ([feast-dev#7116](feast-dev#7116)) ([b1f2168](feast-dev@b1f2168))
* Validate entity column types in feast dbt import ([feast-dev#7067](feast-dev#7067)) ([abca80c](feast-dev@abca80c)), closes [feast-dev#5876](feast-dev#5876)
* Warm up the whole DynamoDB connection pool ([feast-dev#6985](feast-dev#6985)) ([8ce548c](feast-dev@8ce548c))

### Features

* Add --gc-freeze option to feast serve ([feast-dev#7089](feast-dev#7089)) ([aad0c15](feast-dev@aad0c15))
* Add native complex types and scalar index tags to the Milvus online store ([feast-dev#7120](feast-dev#7120)) ([f7068d6](feast-dev@f7068d6))
* Add num_partitions and partition_key_isolation to the Milvus online store ([feast-dev#7083](feast-dev#7083)) ([68de84e](feast-dev@68de84e)), closes [feast-dev#6940](feast-dev#6940)
* Add Pinecone online store integration ([feast-dev#6585](feast-dev#6585)) ([7a84f2a](feast-dev@7a84f2a))
* Add PolarisSource and PolarisProvider for Apache Polaris ([feast-dev#7006](feast-dev#7006)) ([386759c](feast-dev@386759c))
* Add row deletion to the Milvus online store ([feast-dev#7082](feast-dev#7082)) ([d3e2041](feast-dev@d3e2041))
* Add upstream FeatureView lineage support for PushSource ([feast-dev#6853](feast-dev#6853)) ([c9b9d72](feast-dev@c9b9d72))
* Optional nameless pre-computed vectors with a schema fingerprint ([feast-dev#7077](feast-dev#7077)) ([9420fac](feast-dev@9420fac))

### Performance Improvements

* Scan Lance sources lazily in the DuckDB offline store ([1299e4e](feast-dev@1299e4e))

Signed-off-by: Shuchu Han <[email protected]>
@rhods-devops-app rhods-devops-app Bot mentioned this pull request Nov 12, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shuchu shuchu shuchu approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

dbt integration: Validate entity column data type is appropriate

3 participants

@NopeCharon @codecov-commenter @shuchu