Skip to content

fix: Use date-only bounds for partition filters built from string timestamps - #7116

Merged
shuchu merged 2 commits into
feast-dev:masterfrom
sohailshk:fix/athena-partition-string-date
Nov 10, 2026
Merged

fix: Use date-only bounds for partition filters built from string timestamps#7116
shuchu merged 2 commits into
feast-dev:masterfrom
sohailshk:fix/athena-partition-string-date

Conversation

@sohailshk

@sohailshk sohailshk commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

get_timestamp_filter_sql formats datetime bounds as dates for the date_partition_column predicate, but returns string bounds unchanged. AthenaOfflineStore.pull_all_from_table_or_query passes strings such as 2026-02-01 01:00:00.000, so the generated partition predicate was dt >= '2026-02-01 01:00:00.000' AND dt <= '2026-02-02 02:00:00.000'. A partition value '2026-02-01' sorts before that bound, so the first partition day was dropped. String bounds are now truncated to their date part, the same way the date_func cast style already does.

Adds test_pull_all_partition_filter_uses_dates_only, which builds the Athena query for a source with a partition column and checks the partition predicate. It fails on master and passes with the change.

Which issue(s) this PR fixes:

N/A (found while reading the Athena offline store; no existing issue)

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

Misc

pytest tests/unit/infra/offline_stores: 263 passed, 20 skipped; 2 test_ibis.py failures (sqlglot TypeError) also occur on unmodified master.

@sohailshk
fix: Use date-only bounds for partition filters built from string tim…
c7c5ef6 
…estamps

get_timestamp_filter_sql formats datetime bounds as dates for the date_partition_column predicate, but returned string bounds unchanged. Athena's pull_all_from_table_or_query passes strings such as '2026-02-01 01:00:00.000', so the partition predicate became dt >= '2026-02-01 01:00:00.000'. Against a 'YYYY-MM-DD' partition value that compares false, which drops the whole first day. Truncate string bounds to their date part.

Signed-off-by: sohailshk <[email protected]>
@sohailshk
sohailshk requested review from a team as code owners November 9, 2026 10:03
@sohailshk
sohailshk requested review from ejscribner, ntkathole and robhowley and removed request for a team November 9, 2026 10:03
patelchaitany
patelchaitany approved these changes Nov 9, 2026
View reviewed changes

@patelchaitany patelchaitany left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Confirmed the regression test fails before the fix and passes after; all 37 targeted Athena/BigQuery tests and local DuckDB boundary/timezone checks pass.

@shuchu
shuchu merged commit b1f2168 into feast-dev:master Nov 10, 2026
2 checks passed
@patelchaitany patelchaitany mentioned this pull request Nov 10, 2026
Merged
franciscojavierarceo pushed a commit that referenced this pull request Nov 10, 2026
@feast-ci-bot
chore(release): release 0.68.0
fb5e5d5 
# [0.68.0](v0.67.0...v0.68.0) (2026-11-10)

### Bug Fixes

* add watch verb on clusterroles and clusterrolebindings for feast-operator ([#7062](#7062)) ([b6bdda4](b6bdda4))
* Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([#6993](#6993)) ([f416309](f416309))
* Break event timestamp ties by created timestamp in Milvus write batches ([#7084](#7084)) ([bbaca76](bbaca76))
* Break event timestamp ties by created timestamp in the Dask offline store ([#6952](#6952)) ([eb93b08](eb93b08))
* Build the Go feature server image in two stages ([1e5e90c](1e5e90c)), closes [#7060](#7060)
* Fall back to the serialized body when UDF source misses globals ([#6816](#6816)) ([f61079f](f61079f))
* Install patched libbrotli1 to address DSA-6582-1 ([#7096](#7096)) ([d458393](d458393))
* Keep description, tags and nested types of ODFV schema fields ([8ef6e75](8ef6e75))
* Keep entity columns out of StreamFeatureView proto features ([#7033](#7033)) ([d520584](d520584))
* Keep the metrics server running when a Gunicorn worker exits ([#7012](#7012)) ([68c5d54](68c5d54)), closes [#7011](#7011)
* Key registry proto caches on the proto, not its id() ([#7085](#7085)) ([147adad](147adad))
* Log ODFV materialization skips instead of printing them ([#7117](#7117)) ([8fdcee7](8fdcee7))
* Pin build dependencies of source builds to the CI lock ([#7104](#7104)) ([dec2f24](dec2f24))
* Populate spec.project in registry REST feature view responses (#… ([#7037](#7037)) ([e304a84](e304a84))
* Preserve colliding source timestamps in Dask retrieval ([#7071](#7071)) ([1396c88](1396c88))
* Preserve Dask historical requests without eligible features ([#7000](#7000)) ([fff09f7](fff09f7))
* Read None elements of UUID and Decimal lists as None ([#7039](#7039)) ([03f30fe](03f30fe))
* Retire the support of Python3.10 ([#7101](#7101)) ([a56766e](a56766e)), closes [#7104](#7104) [#7089](#7089) [#7106](#7106) [#7105](#7105)
* Serialize UUID and TimeUuid entity keys ([#7032](#7032)) ([0c845a1](0c845a1))
* Stop the Ray engine from hiding materialization errors ([#7026](#7026)) ([7903e2e](7903e2e)), closes [#7024](#7024)
* **ui:** stop treating the monitoring baseline as the latest window ([#7056](#7056)) ([0f01802](0f01802)), closes [#7054](#7054)
* Use date-only bounds for partition filters built from string timestamps ([#7116](#7116)) ([b1f2168](b1f2168))
* Validate entity column types in feast dbt import ([#7067](#7067)) ([abca80c](abca80c)), closes [#5876](#5876)
* Warm up the whole DynamoDB connection pool ([#6985](#6985)) ([8ce548c](8ce548c))

### Features

* Add --gc-freeze option to feast serve ([#7089](#7089)) ([aad0c15](aad0c15))
* Add native complex types and scalar index tags to the Milvus online store ([#7120](#7120)) ([f7068d6](f7068d6))
* Add num_partitions and partition_key_isolation to the Milvus online store ([#7083](#7083)) ([68de84e](68de84e)), closes [#6940](#6940)
* Add Pinecone online store integration ([#6585](#6585)) ([7a84f2a](7a84f2a))
* Add PolarisSource and PolarisProvider for Apache Polaris ([#7006](#7006)) ([386759c](386759c))
* Add row deletion to the Milvus online store ([#7082](#7082)) ([d3e2041](d3e2041))
* Add upstream FeatureView lineage support for PushSource ([#6853](#6853)) ([c9b9d72](c9b9d72))
* Optional nameless pre-computed vectors with a schema fingerprint ([#7077](#7077)) ([9420fac](9420fac))

### Performance Improvements

* Scan Lance sources lazily in the DuckDB offline store ([1299e4e](1299e4e))
@patelchaitany patelchaitany mentioned this pull request Nov 11, 2026
Merged
shuchu added a commit to shuchu/feast that referenced this pull request Nov 11, 2026
@sohailshk @shuchu
fix: Use date-only bounds for partition filters built from string tim…
adff45c 
…estamps (feast-dev#7116)

get_timestamp_filter_sql formats datetime bounds as dates for the date_partition_column predicate, but returned string bounds unchanged. Athena's pull_all_from_table_or_query passes strings such as '2026-02-01 01:00:00.000', so the partition predicate became dt >= '2026-02-01 01:00:00.000'. Against a 'YYYY-MM-DD' partition value that compares false, which drops the whole first day. Truncate string bounds to their date part.

Signed-off-by: sohailshk <[email protected]>
Co-authored-by: Shuchu Han <[email protected]>
Signed-off-by: Shuchu Han <[email protected]>
shuchu pushed a commit to shuchu/feast that referenced this pull request Nov 11, 2026
@feast-ci-bot @shuchu
chore(release): release 0.68.0
5b94c9e 
# [0.68.0](feast-dev/feast@v0.67.0...v0.68.0) (2026-11-10)

### Bug Fixes

* add watch verb on clusterroles and clusterrolebindings for feast-operator ([feast-dev#7062](feast-dev#7062)) ([b6bdda4](feast-dev@b6bdda4))
* Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([feast-dev#6993](feast-dev#6993)) ([f416309](feast-dev@f416309))
* Break event timestamp ties by created timestamp in Milvus write batches ([feast-dev#7084](feast-dev#7084)) ([bbaca76](feast-dev@bbaca76))
* Break event timestamp ties by created timestamp in the Dask offline store ([feast-dev#6952](feast-dev#6952)) ([eb93b08](feast-dev@eb93b08))
* Build the Go feature server image in two stages ([1e5e90c](feast-dev@1e5e90c)), closes [feast-dev#7060](feast-dev#7060)
* Fall back to the serialized body when UDF source misses globals ([feast-dev#6816](feast-dev#6816)) ([f61079f](feast-dev@f61079f))
* Install patched libbrotli1 to address DSA-6582-1 ([feast-dev#7096](feast-dev#7096)) ([d458393](feast-dev@d458393))
* Keep description, tags and nested types of ODFV schema fields ([8ef6e75](feast-dev@8ef6e75))
* Keep entity columns out of StreamFeatureView proto features ([feast-dev#7033](feast-dev#7033)) ([d520584](feast-dev@d520584))
* Keep the metrics server running when a Gunicorn worker exits ([feast-dev#7012](feast-dev#7012)) ([68c5d54](feast-dev@68c5d54)), closes [feast-dev#7011](feast-dev#7011)
* Key registry proto caches on the proto, not its id() ([feast-dev#7085](feast-dev#7085)) ([147adad](feast-dev@147adad))
* Log ODFV materialization skips instead of printing them ([feast-dev#7117](feast-dev#7117)) ([8fdcee7](feast-dev@8fdcee7))
* Pin build dependencies of source builds to the CI lock ([feast-dev#7104](feast-dev#7104)) ([dec2f24](feast-dev@dec2f24))
* Populate spec.project in registry REST feature view responses (#… ([feast-dev#7037](feast-dev#7037)) ([e304a84](feast-dev@e304a84))
* Preserve colliding source timestamps in Dask retrieval ([feast-dev#7071](feast-dev#7071)) ([1396c88](feast-dev@1396c88))
* Preserve Dask historical requests without eligible features ([feast-dev#7000](feast-dev#7000)) ([fff09f7](feast-dev@fff09f7))
* Read None elements of UUID and Decimal lists as None ([feast-dev#7039](feast-dev#7039)) ([03f30fe](feast-dev@03f30fe))
* Retire the support of Python3.10 ([feast-dev#7101](feast-dev#7101)) ([a56766e](feast-dev@a56766e)), closes [feast-dev#7104](feast-dev#7104) [feast-dev#7089](feast-dev#7089) [feast-dev#7106](feast-dev#7106) [feast-dev#7105](feast-dev#7105)
* Serialize UUID and TimeUuid entity keys ([feast-dev#7032](feast-dev#7032)) ([0c845a1](feast-dev@0c845a1))
* Stop the Ray engine from hiding materialization errors ([feast-dev#7026](feast-dev#7026)) ([7903e2e](feast-dev@7903e2e)), closes [feast-dev#7024](feast-dev#7024)
* **ui:** stop treating the monitoring baseline as the latest window ([feast-dev#7056](feast-dev#7056)) ([0f01802](feast-dev@0f01802)), closes [feast-dev#7054](feast-dev#7054)
* Use date-only bounds for partition filters built from string timestamps ([feast-dev#7116](feast-dev#7116)) ([b1f2168](feast-dev@b1f2168))
* Validate entity column types in feast dbt import ([feast-dev#7067](feast-dev#7067)) ([abca80c](feast-dev@abca80c)), closes [feast-dev#5876](feast-dev#5876)
* Warm up the whole DynamoDB connection pool ([feast-dev#6985](feast-dev#6985)) ([8ce548c](feast-dev@8ce548c))

### Features

* Add --gc-freeze option to feast serve ([feast-dev#7089](feast-dev#7089)) ([aad0c15](feast-dev@aad0c15))
* Add native complex types and scalar index tags to the Milvus online store ([feast-dev#7120](feast-dev#7120)) ([f7068d6](feast-dev@f7068d6))
* Add num_partitions and partition_key_isolation to the Milvus online store ([feast-dev#7083](feast-dev#7083)) ([68de84e](feast-dev@68de84e)), closes [feast-dev#6940](feast-dev#6940)
* Add Pinecone online store integration ([feast-dev#6585](feast-dev#6585)) ([7a84f2a](feast-dev@7a84f2a))
* Add PolarisSource and PolarisProvider for Apache Polaris ([feast-dev#7006](feast-dev#7006)) ([386759c](feast-dev@386759c))
* Add row deletion to the Milvus online store ([feast-dev#7082](feast-dev#7082)) ([d3e2041](feast-dev@d3e2041))
* Add upstream FeatureView lineage support for PushSource ([feast-dev#6853](feast-dev#6853)) ([c9b9d72](feast-dev@c9b9d72))
* Optional nameless pre-computed vectors with a schema fingerprint ([feast-dev#7077](feast-dev#7077)) ([9420fac](feast-dev@9420fac))

### Performance Improvements

* Scan Lance sources lazily in the DuckDB offline store ([1299e4e](feast-dev@1299e4e))

Signed-off-by: Shuchu Han <[email protected]>
@rhods-devops-app rhods-devops-app Bot mentioned this pull request Nov 12, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ejscribner ejscribner Awaiting requested review from ejscribner ejscribner is a code owner automatically assigned from feast-dev/maintainers
@robhowley robhowley Awaiting requested review from robhowley robhowley is a code owner automatically assigned from feast-dev/maintainers
@ntkathole ntkathole Awaiting requested review from ntkathole ntkathole is a code owner automatically assigned from feast-dev/maintainers
1 more reviewer
@patelchaitany patelchaitany patelchaitany approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sohailshk @patelchaitany @shuchu