feat: Support Milvus partition keys in the Milvus online store by simonhearne · Pull Request #6940 · feast-dev/feast

simonhearne · 2026-10-05T09:00:31Z

What this PR does / why we need it:

Multi-tenant feature views, such as a catalogue shared by many brands, benefit from Milvus
partition keys: a search filtered on the key only scans the matching partitions.

The partition key can be set per feature view with a tag, or for the whole store:

Feature view tag milvus.partition_key: <field>.
Store config partition_key: <field>, applied to every feature view that has that field.

The tag takes precedence. The field must be stored as VARCHAR or INT64; other types raise a clear
error. The setting only applies when Feast creates a collection: existing collections are left
alone, and Feast logs a warning explaining how to recreate them.

online_store:
  type: milvus
  partition_key: "brand_id"

FeatureView(
    name="products",
    ...,
    tags={"milvus.partition_key": "brand_id"},
)

Which issue(s) this PR fixes:

N/A

Checks

I've made sure the tests are passing.
My commits are signed off (git commit -s)
My PR title follows conventional commits format

Testing Strategy

Unit tests
Integration tests
Manual tests
Testing is not required for this change
Unit (Milvus Lite): the tag and the store config both mark the field as the partition key, and a
filtered search returns only that tenant's rows; the store config is ignored for feature views
without the field; the tag takes precedence over the config.
Unit (mocked client): a missing field and a FLOAT field are rejected; existing collections log a
warning.
Server: the partition key is recorded in the collection schema and filtered search works.

Unit tests run on Milvus Lite 3.2.1 (pymilvus 3.0.2). Server tests in sdk/python/tests/integration/online_store/test_milvus_remote.py are marked integration and skip unless ZILLIZ_URI and ZILLIZ_TOKEN are set; they passed against a local Milvus 2.6.0, a local Milvus 3.0.2 server and against Zilliz Cloud. The existing Milvus unit and universal integration tests pass unchanged.

Misc

Part of a series of Milvus online store improvements for Zilliz Cloud and production Milvus, following #6882, #6895, #6917 and #6921.

codecov-commenter · 2026-10-05T09:04:17Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 96.29630% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 49.06%. Comparing base (6b212f5) to head (345d382).
⚠️ Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
.../infra/online_stores/milvus_online_store/milvus.py	96.29%	0 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6940      +/-   ##
==========================================
- Coverage   49.08%   49.06%   -0.03%     
==========================================
  Files         433      435       +2     
  Lines       54320    54528     +208     
  Branches     7915     7954      +39     
==========================================
+ Hits        26663    26752      +89     
- Misses      25781    25898     +117     
- Partials     1876     1878       +2

Flag	Coverage Δ
go-feature-server	`30.58% <ø> (ø)`
python-unit	`50.44% <96.29%> (-0.04%)`	⬇️

Files with missing lines	Coverage Δ
.../infra/online_stores/milvus_online_store/milvus.py	`76.17% <96.29%> (+3.38%)`	⬆️

... and 12 files with indirect coverage changes

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d15533...345d382. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Multi-tenant feature views, such as a catalogue shared by many brands, benefit from Milvus partition keys: searches filtered on the key only scan the matching partitions. A field can be marked as the partition key with the feature view tag milvus.partition_key, or for every feature view containing the field with the new partition_key store config. The tag takes precedence. The field must be stored as VARCHAR or INT64. Partition keys only apply when a collection is created. Feast logs a warning when an existing collection lacks the configured key. Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Simon Hearne <[email protected]>

…line store (#7083) * feat: Add num_partitions and partition_key_isolation to the Milvus online store A partition key (#6940) comes with two collection settings that Milvus only accepts when the collection is created: the number of partitions, and partition key isolation, which makes a search filtered on the key scan only that key's data. num_partitions and partition_key_isolation are passed to create_collection for feature views that have a partition key. They are ignored for other feature views. When unset nothing is sent, so Milvus keeps its defaults. Signed-off-by: jiwidi <[email protected]> * ci: Re-run the checks after a macOS runner failure Signed-off-by: jiwidi <[email protected]> --------- Signed-off-by: jiwidi <[email protected]> Co-authored-by: Shuchu Han <[email protected]>

# [0.68.0](v0.67.0...v0.68.0) (2026-11-10) ### Bug Fixes * add watch verb on clusterroles and clusterrolebindings for feast-operator ([#7062](#7062)) ([b6bdda4](b6bdda4)) * Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([#6993](#6993)) ([f416309](f416309)) * Break event timestamp ties by created timestamp in Milvus write batches ([#7084](#7084)) ([bbaca76](bbaca76)) * Break event timestamp ties by created timestamp in the Dask offline store ([#6952](#6952)) ([eb93b08](eb93b08)) * Build the Go feature server image in two stages ([1e5e90c](1e5e90c)), closes [#7060](#7060) * Fall back to the serialized body when UDF source misses globals ([#6816](#6816)) ([f61079f](f61079f)) * Install patched libbrotli1 to address DSA-6582-1 ([#7096](#7096)) ([d458393](d458393)) * Keep description, tags and nested types of ODFV schema fields ([8ef6e75](8ef6e75)) * Keep entity columns out of StreamFeatureView proto features ([#7033](#7033)) ([d520584](d520584)) * Keep the metrics server running when a Gunicorn worker exits ([#7012](#7012)) ([68c5d54](68c5d54)), closes [#7011](#7011) * Key registry proto caches on the proto, not its id() ([#7085](#7085)) ([147adad](147adad)) * Log ODFV materialization skips instead of printing them ([#7117](#7117)) ([8fdcee7](8fdcee7)) * Pin build dependencies of source builds to the CI lock ([#7104](#7104)) ([dec2f24](dec2f24)) * Populate spec.project in registry REST feature view responses (#… ([#7037](#7037)) ([e304a84](e304a84)) * Preserve colliding source timestamps in Dask retrieval ([#7071](#7071)) ([1396c88](1396c88)) * Preserve Dask historical requests without eligible features ([#7000](#7000)) ([fff09f7](fff09f7)) * Read None elements of UUID and Decimal lists as None ([#7039](#7039)) ([03f30fe](03f30fe)) * Retire the support of Python3.10 ([#7101](#7101)) ([a56766e](a56766e)), closes [#7104](#7104) [#7089](#7089) [#7106](#7106) [#7105](#7105) * Serialize UUID and TimeUuid entity keys ([#7032](#7032)) ([0c845a1](0c845a1)) * Stop the Ray engine from hiding materialization errors ([#7026](#7026)) ([7903e2e](7903e2e)), closes [#7024](#7024) * **ui:** stop treating the monitoring baseline as the latest window ([#7056](#7056)) ([0f01802](0f01802)), closes [#7054](#7054) * Use date-only bounds for partition filters built from string timestamps ([#7116](#7116)) ([b1f2168](b1f2168)) * Validate entity column types in feast dbt import ([#7067](#7067)) ([abca80c](abca80c)), closes [#5876](#5876) * Warm up the whole DynamoDB connection pool ([#6985](#6985)) ([8ce548c](8ce548c)) ### Features * Add --gc-freeze option to feast serve ([#7089](#7089)) ([aad0c15](aad0c15)) * Add native complex types and scalar index tags to the Milvus online store ([#7120](#7120)) ([f7068d6](f7068d6)) * Add num_partitions and partition_key_isolation to the Milvus online store ([#7083](#7083)) ([68de84e](68de84e)), closes [#6940](#6940) * Add Pinecone online store integration ([#6585](#6585)) ([7a84f2a](7a84f2a)) * Add PolarisSource and PolarisProvider for Apache Polaris ([#7006](#7006)) ([386759c](386759c)) * Add row deletion to the Milvus online store ([#7082](#7082)) ([d3e2041](d3e2041)) * Add upstream FeatureView lineage support for PushSource ([#6853](#6853)) ([c9b9d72](c9b9d72)) * Optional nameless pre-computed vectors with a schema fingerprint ([#7077](#7077)) ([9420fac](9420fac)) ### Performance Improvements * Scan Lance sources lazily in the DuckDB offline store ([1299e4e](1299e4e))

# [0.68.0](feast-dev/feast@v0.67.0...v0.68.0) (2026-11-10) ### Bug Fixes * add watch verb on clusterroles and clusterrolebindings for feast-operator ([feast-dev#7062](feast-dev#7062)) ([b6bdda4](feast-dev@b6bdda4)) * Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([feast-dev#6993](feast-dev#6993)) ([f416309](feast-dev@f416309)) * Break event timestamp ties by created timestamp in Milvus write batches ([feast-dev#7084](feast-dev#7084)) ([bbaca76](feast-dev@bbaca76)) * Break event timestamp ties by created timestamp in the Dask offline store ([feast-dev#6952](feast-dev#6952)) ([eb93b08](feast-dev@eb93b08)) * Build the Go feature server image in two stages ([1e5e90c](feast-dev@1e5e90c)), closes [feast-dev#7060](feast-dev#7060) * Fall back to the serialized body when UDF source misses globals ([feast-dev#6816](feast-dev#6816)) ([f61079f](feast-dev@f61079f)) * Install patched libbrotli1 to address DSA-6582-1 ([feast-dev#7096](feast-dev#7096)) ([d458393](feast-dev@d458393)) * Keep description, tags and nested types of ODFV schema fields ([8ef6e75](feast-dev@8ef6e75)) * Keep entity columns out of StreamFeatureView proto features ([feast-dev#7033](feast-dev#7033)) ([d520584](feast-dev@d520584)) * Keep the metrics server running when a Gunicorn worker exits ([feast-dev#7012](feast-dev#7012)) ([68c5d54](feast-dev@68c5d54)), closes [feast-dev#7011](feast-dev#7011) * Key registry proto caches on the proto, not its id() ([feast-dev#7085](feast-dev#7085)) ([147adad](feast-dev@147adad)) * Log ODFV materialization skips instead of printing them ([feast-dev#7117](feast-dev#7117)) ([8fdcee7](feast-dev@8fdcee7)) * Pin build dependencies of source builds to the CI lock ([feast-dev#7104](feast-dev#7104)) ([dec2f24](feast-dev@dec2f24)) * Populate spec.project in registry REST feature view responses (#… ([feast-dev#7037](feast-dev#7037)) ([e304a84](feast-dev@e304a84)) * Preserve colliding source timestamps in Dask retrieval ([feast-dev#7071](feast-dev#7071)) ([1396c88](feast-dev@1396c88)) * Preserve Dask historical requests without eligible features ([feast-dev#7000](feast-dev#7000)) ([fff09f7](feast-dev@fff09f7)) * Read None elements of UUID and Decimal lists as None ([feast-dev#7039](feast-dev#7039)) ([03f30fe](feast-dev@03f30fe)) * Retire the support of Python3.10 ([feast-dev#7101](feast-dev#7101)) ([a56766e](feast-dev@a56766e)), closes [feast-dev#7104](feast-dev#7104) [feast-dev#7089](feast-dev#7089) [feast-dev#7106](feast-dev#7106) [feast-dev#7105](feast-dev#7105) * Serialize UUID and TimeUuid entity keys ([feast-dev#7032](feast-dev#7032)) ([0c845a1](feast-dev@0c845a1)) * Stop the Ray engine from hiding materialization errors ([feast-dev#7026](feast-dev#7026)) ([7903e2e](feast-dev@7903e2e)), closes [feast-dev#7024](feast-dev#7024) * **ui:** stop treating the monitoring baseline as the latest window ([feast-dev#7056](feast-dev#7056)) ([0f01802](feast-dev@0f01802)), closes [feast-dev#7054](feast-dev#7054) * Use date-only bounds for partition filters built from string timestamps ([feast-dev#7116](feast-dev#7116)) ([b1f2168](feast-dev@b1f2168)) * Validate entity column types in feast dbt import ([feast-dev#7067](feast-dev#7067)) ([abca80c](feast-dev@abca80c)), closes [feast-dev#5876](feast-dev#5876) * Warm up the whole DynamoDB connection pool ([feast-dev#6985](feast-dev#6985)) ([8ce548c](feast-dev@8ce548c)) ### Features * Add --gc-freeze option to feast serve ([feast-dev#7089](feast-dev#7089)) ([aad0c15](feast-dev@aad0c15)) * Add native complex types and scalar index tags to the Milvus online store ([feast-dev#7120](feast-dev#7120)) ([f7068d6](feast-dev@f7068d6)) * Add num_partitions and partition_key_isolation to the Milvus online store ([feast-dev#7083](feast-dev#7083)) ([68de84e](feast-dev@68de84e)), closes [feast-dev#6940](feast-dev#6940) * Add Pinecone online store integration ([feast-dev#6585](feast-dev#6585)) ([7a84f2a](feast-dev@7a84f2a)) * Add PolarisSource and PolarisProvider for Apache Polaris ([feast-dev#7006](feast-dev#7006)) ([386759c](feast-dev@386759c)) * Add row deletion to the Milvus online store ([feast-dev#7082](feast-dev#7082)) ([d3e2041](feast-dev@d3e2041)) * Add upstream FeatureView lineage support for PushSource ([feast-dev#6853](feast-dev#6853)) ([c9b9d72](feast-dev@c9b9d72)) * Optional nameless pre-computed vectors with a schema fingerprint ([feast-dev#7077](feast-dev#7077)) ([9420fac](feast-dev@9420fac)) ### Performance Improvements * Scan Lance sources lazily in the DuckDB offline store ([1299e4e](feast-dev@1299e4e)) Signed-off-by: Shuchu Han <[email protected]>

simonhearne requested a review from a team as a code owner October 5, 2026 09:00

ntkathole added the ok-to-test label Oct 5, 2026

Rodrigo-Palma mentioned this pull request Oct 5, 2026

fix: Keep tiling, offline and stream engine settings when copying a StreamFeatureView #6924

Merged

4 of 7 tasks

ntkathole force-pushed the feat/milvus-partition-key branch from 2b6e047 to 992f393 Compare October 7, 2026 04:47

ntkathole approved these changes Oct 7, 2026

View reviewed changes

ntkathole force-pushed the feat/milvus-partition-key branch from 992f393 to 345d382 Compare October 7, 2026 06:48

ntkathole merged commit 0ed3285 into feast-dev:master Oct 7, 2026
18 of 23 checks passed

simonhearne mentioned this pull request Oct 16, 2026

feat: Add native_numeric_types to the Milvus online store #7025

Closed

4 of 7 tasks

This was referenced Oct 28, 2026

feat: Add row deletion to the Milvus online store #7082

Merged

feat: Add num_partitions and partition_key_isolation to the Milvus online store #7083

Merged

fix: Break event timestamp ties by created timestamp in Milvus write batches #7084

Merged

This was referenced Nov 9, 2026

feat: Add a schema check and mutation timeout to the Milvus online store #7118

Merged

fix: Round-trip every Feast type and null in the Milvus online store #7119

Closed

feat: Add native complex types and scalar index tags to the Milvus online store #7120

Merged

Navigation Menu

Repository navigation

feat: Support Milvus partition keys in the Milvus online store - #6940

feat: Support Milvus partition keys in the Milvus online store#6940
ntkathole merged 1 commit into
feast-dev:masterfrom
simonhearne:feat/milvus-partition-key

simonhearne commented Oct 5, 2026

Uh oh!

codecov-commenter commented Oct 5, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

simonhearne commented Oct 5, 2026

What this PR does / why we need it:

Which issue(s) this PR fixes:

Checks

Testing Strategy

Misc

Uh oh!

codecov-commenter commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov-commenter commented Oct 5, 2026 •

edited

Loading