feat: Add num_partitions and partition_key_isolation to the Milvus online store by jiwidi · Pull Request #7083 · feast-dev/feast

jiwidi · 2026-10-28T11:05:11Z

What this PR does / why we need it:

#6940 added partition keys. Milvus has two per-collection settings that go with them and that can
only be set when the collection is created: the number of partitions, and
partition key isolation,
which makes a search filtered on the key scan only that key's data. Like key_ttl_seconds in the
Redis store, they are settings the store applies when it creates data and cannot change later.

Field	Behaviour
`num_partitions`	Number of partitions of a new collection that has a partition key.
`partition_key_isolation`	Sets the `partitionkey.isolation` property of a new collection that has a partition key. Milvus supports it with the HNSW index.

Both are ignored for feature views without a partition key, and both only apply when Feast creates
the collection, like partition_key itself. When unset nothing is sent, so Milvus keeps its
defaults. Milvus Lite ignores num_partitions.

online_store:
  type: milvus
  partition_key: "brand_id"
  num_partitions: 32
  partition_key_isolation: true
  index_type: "HNSW"

Which issue(s) this PR fixes:

N/A

Checks

I've made sure the tests are passing.
My commits are signed off (git commit -s)
My PR title follows conventional commits format

Testing Strategy

Unit tests
Integration tests
Manual tests
Testing is not required for this change
Unit (mocked client): both settings are sent for a feature view with a partition key and not
for one without; nothing is sent when unset; num_partitions: 0 is rejected.
Server: the collection reports the partition count and the isolation property, and a filtered
search returns only that tenant's rows.

Unit tests run on Milvus Lite 3.2.1 (pymilvus 3.0.2). The server test in
sdk/python/tests/integration/online_store/test_milvus_remote.py passes against a local
Milvus 3.0.2 server. The existing Milvus unit and server tests pass unchanged.

Misc

Part of a stack that adds to the Milvus online store the update and delete functionality the Redis
online store has, following #6882, #6895, #6917, #6921 and #6940. The other PRs of the stack are
#7082 (row deletion) and #7084 (tie-break in write batches). mutation_timeout and a schema mismatch check follow once these merge.

…line store A partition key (feast-dev#6940) comes with two collection settings that Milvus only accepts when the collection is created: the number of partitions, and partition key isolation, which makes a search filtered on the key scan only that key's data. num_partitions and partition_key_isolation are passed to create_collection for feature views that have a partition key. They are ignored for other feature views. When unset nothing is sent, so Milvus keeps its defaults. Signed-off-by: jiwidi <[email protected]>

codecov-commenter · 2026-10-28T11:12:05Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 51.32%. Comparing base (fde9529) to head (4426d19).
⚠️ Report is 3 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7083      +/-   ##
==========================================
+ Coverage   51.30%   51.32%   +0.01%     
==========================================
  Files         458      458              
  Lines       57423    57440      +17     
  Branches     8417     8421       +4     
==========================================
+ Hits        29463    29480      +17     
  Misses      25950    25950              
  Partials     2010     2010

Flag	Coverage Δ
go-feature-server	`30.58% <ø> (ø)`
python-unit	`52.79% <100.00%> (+0.01%)`	⬆️

Files with missing lines	Coverage Δ
.../infra/online_stores/milvus_online_store/milvus.py	`77.37% <100.00%> (+0.72%)`	⬆️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a56766e...4426d19. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

shuchu

lgtm

shuchu · 2026-11-04T03:23:52Z

@jiwidi the PR is out-of-date, please update it.

…tion-options Signed-off-by: jiwidi <[email protected]> # Conflicts: # sdk/python/tests/integration/online_store/test_milvus_remote.py # sdk/python/tests/unit/infra/online_store/test_milvus_online_store.py

jiwidi · 2026-11-04T13:58:36Z

@jiwidi the PR is out-of-date, please update it.

hi, did that now, seems like failing to a dep issue that #7104 #7104 introduced. Will reupdate with main when that lands in so test can run!

Signed-off-by: jiwidi <[email protected]>

jiwidi · 2026-11-06T12:46:38Z

@shuchu should be ready to merge now!

# [0.68.0](v0.67.0...v0.68.0) (2026-11-10) ### Bug Fixes * add watch verb on clusterroles and clusterrolebindings for feast-operator ([#7062](#7062)) ([b6bdda4](b6bdda4)) * Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([#6993](#6993)) ([f416309](f416309)) * Break event timestamp ties by created timestamp in Milvus write batches ([#7084](#7084)) ([bbaca76](bbaca76)) * Break event timestamp ties by created timestamp in the Dask offline store ([#6952](#6952)) ([eb93b08](eb93b08)) * Build the Go feature server image in two stages ([1e5e90c](1e5e90c)), closes [#7060](#7060) * Fall back to the serialized body when UDF source misses globals ([#6816](#6816)) ([f61079f](f61079f)) * Install patched libbrotli1 to address DSA-6582-1 ([#7096](#7096)) ([d458393](d458393)) * Keep description, tags and nested types of ODFV schema fields ([8ef6e75](8ef6e75)) * Keep entity columns out of StreamFeatureView proto features ([#7033](#7033)) ([d520584](d520584)) * Keep the metrics server running when a Gunicorn worker exits ([#7012](#7012)) ([68c5d54](68c5d54)), closes [#7011](#7011) * Key registry proto caches on the proto, not its id() ([#7085](#7085)) ([147adad](147adad)) * Log ODFV materialization skips instead of printing them ([#7117](#7117)) ([8fdcee7](8fdcee7)) * Pin build dependencies of source builds to the CI lock ([#7104](#7104)) ([dec2f24](dec2f24)) * Populate spec.project in registry REST feature view responses (#… ([#7037](#7037)) ([e304a84](e304a84)) * Preserve colliding source timestamps in Dask retrieval ([#7071](#7071)) ([1396c88](1396c88)) * Preserve Dask historical requests without eligible features ([#7000](#7000)) ([fff09f7](fff09f7)) * Read None elements of UUID and Decimal lists as None ([#7039](#7039)) ([03f30fe](03f30fe)) * Retire the support of Python3.10 ([#7101](#7101)) ([a56766e](a56766e)), closes [#7104](#7104) [#7089](#7089) [#7106](#7106) [#7105](#7105) * Serialize UUID and TimeUuid entity keys ([#7032](#7032)) ([0c845a1](0c845a1)) * Stop the Ray engine from hiding materialization errors ([#7026](#7026)) ([7903e2e](7903e2e)), closes [#7024](#7024) * **ui:** stop treating the monitoring baseline as the latest window ([#7056](#7056)) ([0f01802](0f01802)), closes [#7054](#7054) * Use date-only bounds for partition filters built from string timestamps ([#7116](#7116)) ([b1f2168](b1f2168)) * Validate entity column types in feast dbt import ([#7067](#7067)) ([abca80c](abca80c)), closes [#5876](#5876) * Warm up the whole DynamoDB connection pool ([#6985](#6985)) ([8ce548c](8ce548c)) ### Features * Add --gc-freeze option to feast serve ([#7089](#7089)) ([aad0c15](aad0c15)) * Add native complex types and scalar index tags to the Milvus online store ([#7120](#7120)) ([f7068d6](f7068d6)) * Add num_partitions and partition_key_isolation to the Milvus online store ([#7083](#7083)) ([68de84e](68de84e)), closes [#6940](#6940) * Add Pinecone online store integration ([#6585](#6585)) ([7a84f2a](7a84f2a)) * Add PolarisSource and PolarisProvider for Apache Polaris ([#7006](#7006)) ([386759c](386759c)) * Add row deletion to the Milvus online store ([#7082](#7082)) ([d3e2041](d3e2041)) * Add upstream FeatureView lineage support for PushSource ([#6853](#6853)) ([c9b9d72](c9b9d72)) * Optional nameless pre-computed vectors with a schema fingerprint ([#7077](#7077)) ([9420fac](9420fac)) ### Performance Improvements * Scan Lance sources lazily in the DuckDB offline store ([1299e4e](1299e4e))

# [0.68.0](feast-dev/feast@v0.67.0...v0.68.0) (2026-11-10) ### Bug Fixes * add watch verb on clusterroles and clusterrolebindings for feast-operator ([feast-dev#7062](feast-dev#7062)) ([b6bdda4](feast-dev@b6bdda4)) * Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([feast-dev#6993](feast-dev#6993)) ([f416309](feast-dev@f416309)) * Break event timestamp ties by created timestamp in Milvus write batches ([feast-dev#7084](feast-dev#7084)) ([bbaca76](feast-dev@bbaca76)) * Break event timestamp ties by created timestamp in the Dask offline store ([feast-dev#6952](feast-dev#6952)) ([eb93b08](feast-dev@eb93b08)) * Build the Go feature server image in two stages ([1e5e90c](feast-dev@1e5e90c)), closes [feast-dev#7060](feast-dev#7060) * Fall back to the serialized body when UDF source misses globals ([feast-dev#6816](feast-dev#6816)) ([f61079f](feast-dev@f61079f)) * Install patched libbrotli1 to address DSA-6582-1 ([feast-dev#7096](feast-dev#7096)) ([d458393](feast-dev@d458393)) * Keep description, tags and nested types of ODFV schema fields ([8ef6e75](feast-dev@8ef6e75)) * Keep entity columns out of StreamFeatureView proto features ([feast-dev#7033](feast-dev#7033)) ([d520584](feast-dev@d520584)) * Keep the metrics server running when a Gunicorn worker exits ([feast-dev#7012](feast-dev#7012)) ([68c5d54](feast-dev@68c5d54)), closes [feast-dev#7011](feast-dev#7011) * Key registry proto caches on the proto, not its id() ([feast-dev#7085](feast-dev#7085)) ([147adad](feast-dev@147adad)) * Log ODFV materialization skips instead of printing them ([feast-dev#7117](feast-dev#7117)) ([8fdcee7](feast-dev@8fdcee7)) * Pin build dependencies of source builds to the CI lock ([feast-dev#7104](feast-dev#7104)) ([dec2f24](feast-dev@dec2f24)) * Populate spec.project in registry REST feature view responses (#… ([feast-dev#7037](feast-dev#7037)) ([e304a84](feast-dev@e304a84)) * Preserve colliding source timestamps in Dask retrieval ([feast-dev#7071](feast-dev#7071)) ([1396c88](feast-dev@1396c88)) * Preserve Dask historical requests without eligible features ([feast-dev#7000](feast-dev#7000)) ([fff09f7](feast-dev@fff09f7)) * Read None elements of UUID and Decimal lists as None ([feast-dev#7039](feast-dev#7039)) ([03f30fe](feast-dev@03f30fe)) * Retire the support of Python3.10 ([feast-dev#7101](feast-dev#7101)) ([a56766e](feast-dev@a56766e)), closes [feast-dev#7104](feast-dev#7104) [feast-dev#7089](feast-dev#7089) [feast-dev#7106](feast-dev#7106) [feast-dev#7105](feast-dev#7105) * Serialize UUID and TimeUuid entity keys ([feast-dev#7032](feast-dev#7032)) ([0c845a1](feast-dev@0c845a1)) * Stop the Ray engine from hiding materialization errors ([feast-dev#7026](feast-dev#7026)) ([7903e2e](feast-dev@7903e2e)), closes [feast-dev#7024](feast-dev#7024) * **ui:** stop treating the monitoring baseline as the latest window ([feast-dev#7056](feast-dev#7056)) ([0f01802](feast-dev@0f01802)), closes [feast-dev#7054](feast-dev#7054) * Use date-only bounds for partition filters built from string timestamps ([feast-dev#7116](feast-dev#7116)) ([b1f2168](feast-dev@b1f2168)) * Validate entity column types in feast dbt import ([feast-dev#7067](feast-dev#7067)) ([abca80c](feast-dev@abca80c)), closes [feast-dev#5876](feast-dev#5876) * Warm up the whole DynamoDB connection pool ([feast-dev#6985](feast-dev#6985)) ([8ce548c](feast-dev@8ce548c)) ### Features * Add --gc-freeze option to feast serve ([feast-dev#7089](feast-dev#7089)) ([aad0c15](feast-dev@aad0c15)) * Add native complex types and scalar index tags to the Milvus online store ([feast-dev#7120](feast-dev#7120)) ([f7068d6](feast-dev@f7068d6)) * Add num_partitions and partition_key_isolation to the Milvus online store ([feast-dev#7083](feast-dev#7083)) ([68de84e](feast-dev@68de84e)), closes [feast-dev#6940](feast-dev#6940) * Add Pinecone online store integration ([feast-dev#6585](feast-dev#6585)) ([7a84f2a](feast-dev@7a84f2a)) * Add PolarisSource and PolarisProvider for Apache Polaris ([feast-dev#7006](feast-dev#7006)) ([386759c](feast-dev@386759c)) * Add row deletion to the Milvus online store ([feast-dev#7082](feast-dev#7082)) ([d3e2041](feast-dev@d3e2041)) * Add upstream FeatureView lineage support for PushSource ([feast-dev#6853](feast-dev#6853)) ([c9b9d72](feast-dev@c9b9d72)) * Optional nameless pre-computed vectors with a schema fingerprint ([feast-dev#7077](feast-dev#7077)) ([9420fac](feast-dev@9420fac)) ### Performance Improvements * Scan Lance sources lazily in the DuckDB offline store ([1299e4e](feast-dev@1299e4e)) Signed-off-by: Shuchu Han <[email protected]>

jiwidi requested a review from a team as a code owner October 28, 2026 11:05

This was referenced Oct 28, 2026

feat: Add row deletion to the Milvus online store #7082

Merged

fix: Break event timestamp ties by created timestamp in Milvus write batches #7084

Merged

shuchu approved these changes Nov 4, 2026

View reviewed changes

Merge remote-tracking branch 'upstream/master' into feat/milvus-parti…

fe5375a

…tion-options Signed-off-by: jiwidi <[email protected]> # Conflicts: # sdk/python/tests/integration/online_store/test_milvus_remote.py # sdk/python/tests/unit/infra/online_store/test_milvus_online_store.py

jiwidi added 3 commits November 4, 2026 15:13

ci: Re-run the checks after a macOS runner failure

85a2b71

Signed-off-by: jiwidi <[email protected]>

Merge branch 'master' into feat/milvus-partition-options

a024772

Merge branch 'master' into feat/milvus-partition-options

dc3dec5

haoxu0 added the ok-to-test label Nov 6, 2026

Merge branch 'master' into feat/milvus-partition-options

4426d19

shuchu merged commit 68de84e into feast-dev:master Nov 6, 2026
16 of 22 checks passed

This was referenced Nov 9, 2026

feat: Add a schema check and mutation timeout to the Milvus online store #7118

Merged

fix: Round-trip every Feast type and null in the Milvus online store #7119

Closed

feat: Add native complex types and scalar index tags to the Milvus online store #7120

Merged

patelchaitany mentioned this pull request Nov 11, 2026

chore: Sync upstream feast-dev/feast stable (0.68.0) opendatahub-io/feast#291

Merged

rhods-devops-app Bot mentioned this pull request Nov 12, 2026

Automated Sync from main to stable red-hat-data-services/feast#630

Open

Navigation Menu

Repository navigation

feat: Add num_partitions and partition_key_isolation to the Milvus online store - #7083

feat: Add num_partitions and partition_key_isolation to the Milvus online store#7083
shuchu merged 6 commits into
feast-dev:masterfrom
jiwidi:feat/milvus-partition-options

jiwidi commented Oct 28, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Oct 28, 2026 •

edited

Loading

Uh oh!

shuchu left a comment

Uh oh!

shuchu commented Nov 4, 2026

Uh oh!

jiwidi commented Nov 4, 2026

Uh oh!

jiwidi commented Nov 6, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jiwidi commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Which issue(s) this PR fixes:

Checks

Testing Strategy

Misc

Uh oh!

codecov-commenter commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

shuchu left a comment

Choose a reason for hiding this comment

Uh oh!

shuchu commented Nov 4, 2026

Uh oh!

jiwidi commented Nov 4, 2026

Uh oh!

jiwidi commented Nov 6, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jiwidi commented Oct 28, 2026 •

edited

Loading

codecov-commenter commented Oct 28, 2026 •

edited

Loading