feat: Add row deletion to the Milvus online store by jiwidi · Pull Request #7082 · feast-dev/feast

jiwidi · 2026-10-28T11:05:09Z

What this PR does / why we need it:

The Redis online store can remove rows: delete_entity_values removes the rows of an entity and
delete_table removes the rows of a feature view. The Milvus online store has nothing between
upsert and drop_collection, so a row cannot be removed when its entity leaves the data set, for
example a product that is no longer sold.

This PR adds MilvusOnlineStore.online_delete, which deletes by entity key or by filter:

Call	Behaviour
`entity_keys=[...]`	Deletes by primary key, with the same serialized entity key that writes and reads use.
`filters=ComparisonFilter(...)` or `CompoundFilter(...)`	Deletes the rows the filter selects, through the existing `MilvusFilterTranslator`.
`entity_keys=[]`	Deletes nothing.
both or neither	`ValueError`.
a filter with an empty group, at any level	`ValueError`, so a mistake cannot delete more rows than it names.
missing collection	Returns without creating it.

from feast.filter_models import ComparisonFilter

online_store.online_delete(config, feature_view, entity_keys=[entity_key])
online_store.online_delete(
    config, feature_view, filters=ComparisonFilter(type="eq", key="brand_id", value="acme")
)

It works on every collection Feast created, whichever settings created it. No tombstone is kept: a
later write creates the row again, and Feast does not order a delete against writes from other
processes. The method is store-level, like the Redis delete methods; a FeatureStore-level API can
follow once more stores implement it.

Which issue(s) this PR fixes:

N/A

Checks

I've made sure the tests are passing.
My commits are signed off (git commit -s)
My PR title follows conventional commits format

Testing Strategy

Unit tests
Integration tests
Manual tests
Testing is not required for this change
Unit (Milvus Lite): delete by entity key leaves the other rows; delete by filter; a deleted row
can be written again.
Unit (mocked client): both or neither selector and a filter with an empty group are rejected
before any call; a missing collection is neither created nor queried; an empty key list
does not even connect.
Server: delete by key and by filter are visible to the next Strong read; the row can be
written again.

Unit tests run on Milvus Lite 3.2.1 (pymilvus 3.0.2). The server test in
sdk/python/tests/integration/online_store/test_milvus_remote.py passes against a local
Milvus 3.0.2 server. The existing Milvus unit and server tests pass unchanged.

Misc

Part of a stack that adds to the Milvus online store the update and delete functionality the Redis
online store has, following #6882, #6895, #6917, #6921 and #6940. The other PRs of the stack are
#7083 (partition options) and #7084 (tie-break in write batches). mutation_timeout and a schema mismatch check follow once these merge.

The Milvus online store had no operation between upsert and dropping a collection, so a row could not be removed when its entity left the catalogue. The Redis online store can remove rows with delete_entity_values and delete_table. MilvusOnlineStore.online_delete deletes rows by entity key or by filter. It accepts exactly one of the two, deletes nothing for an empty key list, rejects a filter without conditions, and does not create a missing collection. It works on every collection that Feast created. Signed-off-by: jiwidi <[email protected]>

codecov-commenter · 2026-10-28T11:12:57Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 89.47368% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 51.01%. Comparing base (72013d3) to head (dd64a31).
⚠️ Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
.../infra/online_stores/milvus_online_store/milvus.py	89.47%	1 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7082      +/-   ##
==========================================
+ Coverage   50.99%   51.01%   +0.02%     
==========================================
  Files         455      455              
  Lines       56994    57015      +21     
  Branches     8321     8327       +6     
==========================================
+ Hits        29063    29088      +25     
+ Misses      25946    25939       -7     
- Partials     1985     1988       +3

Flag	Coverage Δ
go-feature-server	`30.58% <ø> (ø)`
python-unit	`52.48% <89.47%> (+0.02%)`	⬆️

Files with missing lines	Coverage Δ
.../infra/online_stores/milvus_online_store/milvus.py	`76.64% <89.47%> (+0.47%)`	⬆️

... and 3 files with indirect coverage changes

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 147adad...dd64a31. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: jiwidi <[email protected]>

patelchaitany

Looks good overall. One concern about nested empty filter groups on the delete path, and a small nit.

patelchaitany · 2026-10-29T07:41:18Z

+            expression = MilvusFilterTranslator().translate(filters)
+            if not expression:
+                raise ValueError("The deletion filter must have at least one condition")


This only catches a filter that is empty at the top level. MilvusFilterTranslator drops empty nested groups, so:

CompoundFilter(type="and", filters=[ ComparisonFilter(type="eq", key="brand_id", value="acme"), CompoundFilter(type="or", filters=[]), ])

becomes (brand_id == 'acme') and deletes every acme row. An empty or should match nothing, so this deletes more than the filter asks for. It can happen when the or is built from a list that turns out empty. Since this is a delete, could we reject an empty group anywhere in the tree, not only at the top?

patelchaitany · 2026-10-29T07:41:18Z

+        collection = self._get_or_create_collection(config, table)
+        self.client.delete(collection_name=collection["collection_name"], **selector)


nit: The guard above already returns when the collection is missing, so this never creates one. It is only here to load and cache the collection. A short comment would help, since it reads like "create, then delete". Also, collection["collection_name"] is the same as collection_name.

The Milvus filter translator drops an empty nested group, so a filter such as and(brand_id == 'acme', or()) deleted every acme row instead of none. online_delete now rejects a filter with an empty group at any level before it connects. The delete path loads the collection with _ensure_loaded instead of _get_or_create_collection, which could not create one after the has_collection guard, and reuses the collection name it computed. Signed-off-by: jiwidi <[email protected]>

jiwidi · 2026-10-29T09:56:00Z

Looks good overall. One concern about nested empty filter groups on the delete path, and a small nit.

hi! thank you for review, pushed now a commit to address the bug and nit.

online_delete now walks the filter and rejects an empty group at any level, and your example is in the rejection test. Left the translator as is since reads go through it too, can do that in a followup commit if you think it should raise there as well.
Nit: dropped the _get_or_create_collection call there. It now checks the collection exists, calls _ensure_loaded (delete by filter fails on an unloaded collection on a real server) and uses collection_name directly.

Signed-off-by: jiwidi <[email protected]>

# [0.68.0](v0.67.0...v0.68.0) (2026-11-10) ### Bug Fixes * add watch verb on clusterroles and clusterrolebindings for feast-operator ([#7062](#7062)) ([b6bdda4](b6bdda4)) * Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([#6993](#6993)) ([f416309](f416309)) * Break event timestamp ties by created timestamp in Milvus write batches ([#7084](#7084)) ([bbaca76](bbaca76)) * Break event timestamp ties by created timestamp in the Dask offline store ([#6952](#6952)) ([eb93b08](eb93b08)) * Build the Go feature server image in two stages ([1e5e90c](1e5e90c)), closes [#7060](#7060) * Fall back to the serialized body when UDF source misses globals ([#6816](#6816)) ([f61079f](f61079f)) * Install patched libbrotli1 to address DSA-6582-1 ([#7096](#7096)) ([d458393](d458393)) * Keep description, tags and nested types of ODFV schema fields ([8ef6e75](8ef6e75)) * Keep entity columns out of StreamFeatureView proto features ([#7033](#7033)) ([d520584](d520584)) * Keep the metrics server running when a Gunicorn worker exits ([#7012](#7012)) ([68c5d54](68c5d54)), closes [#7011](#7011) * Key registry proto caches on the proto, not its id() ([#7085](#7085)) ([147adad](147adad)) * Log ODFV materialization skips instead of printing them ([#7117](#7117)) ([8fdcee7](8fdcee7)) * Pin build dependencies of source builds to the CI lock ([#7104](#7104)) ([dec2f24](dec2f24)) * Populate spec.project in registry REST feature view responses (#… ([#7037](#7037)) ([e304a84](e304a84)) * Preserve colliding source timestamps in Dask retrieval ([#7071](#7071)) ([1396c88](1396c88)) * Preserve Dask historical requests without eligible features ([#7000](#7000)) ([fff09f7](fff09f7)) * Read None elements of UUID and Decimal lists as None ([#7039](#7039)) ([03f30fe](03f30fe)) * Retire the support of Python3.10 ([#7101](#7101)) ([a56766e](a56766e)), closes [#7104](#7104) [#7089](#7089) [#7106](#7106) [#7105](#7105) * Serialize UUID and TimeUuid entity keys ([#7032](#7032)) ([0c845a1](0c845a1)) * Stop the Ray engine from hiding materialization errors ([#7026](#7026)) ([7903e2e](7903e2e)), closes [#7024](#7024) * **ui:** stop treating the monitoring baseline as the latest window ([#7056](#7056)) ([0f01802](0f01802)), closes [#7054](#7054) * Use date-only bounds for partition filters built from string timestamps ([#7116](#7116)) ([b1f2168](b1f2168)) * Validate entity column types in feast dbt import ([#7067](#7067)) ([abca80c](abca80c)), closes [#5876](#5876) * Warm up the whole DynamoDB connection pool ([#6985](#6985)) ([8ce548c](8ce548c)) ### Features * Add --gc-freeze option to feast serve ([#7089](#7089)) ([aad0c15](aad0c15)) * Add native complex types and scalar index tags to the Milvus online store ([#7120](#7120)) ([f7068d6](f7068d6)) * Add num_partitions and partition_key_isolation to the Milvus online store ([#7083](#7083)) ([68de84e](68de84e)), closes [#6940](#6940) * Add Pinecone online store integration ([#6585](#6585)) ([7a84f2a](7a84f2a)) * Add PolarisSource and PolarisProvider for Apache Polaris ([#7006](#7006)) ([386759c](386759c)) * Add row deletion to the Milvus online store ([#7082](#7082)) ([d3e2041](d3e2041)) * Add upstream FeatureView lineage support for PushSource ([#6853](#6853)) ([c9b9d72](c9b9d72)) * Optional nameless pre-computed vectors with a schema fingerprint ([#7077](#7077)) ([9420fac](9420fac)) ### Performance Improvements * Scan Lance sources lazily in the DuckDB offline store ([1299e4e](1299e4e))

# [0.68.0](feast-dev/feast@v0.67.0...v0.68.0) (2026-11-10) ### Bug Fixes * add watch verb on clusterroles and clusterrolebindings for feast-operator ([feast-dev#7062](feast-dev#7062)) ([b6bdda4](feast-dev@b6bdda4)) * Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([feast-dev#6993](feast-dev#6993)) ([f416309](feast-dev@f416309)) * Break event timestamp ties by created timestamp in Milvus write batches ([feast-dev#7084](feast-dev#7084)) ([bbaca76](feast-dev@bbaca76)) * Break event timestamp ties by created timestamp in the Dask offline store ([feast-dev#6952](feast-dev#6952)) ([eb93b08](feast-dev@eb93b08)) * Build the Go feature server image in two stages ([1e5e90c](feast-dev@1e5e90c)), closes [feast-dev#7060](feast-dev#7060) * Fall back to the serialized body when UDF source misses globals ([feast-dev#6816](feast-dev#6816)) ([f61079f](feast-dev@f61079f)) * Install patched libbrotli1 to address DSA-6582-1 ([feast-dev#7096](feast-dev#7096)) ([d458393](feast-dev@d458393)) * Keep description, tags and nested types of ODFV schema fields ([8ef6e75](feast-dev@8ef6e75)) * Keep entity columns out of StreamFeatureView proto features ([feast-dev#7033](feast-dev#7033)) ([d520584](feast-dev@d520584)) * Keep the metrics server running when a Gunicorn worker exits ([feast-dev#7012](feast-dev#7012)) ([68c5d54](feast-dev@68c5d54)), closes [feast-dev#7011](feast-dev#7011) * Key registry proto caches on the proto, not its id() ([feast-dev#7085](feast-dev#7085)) ([147adad](feast-dev@147adad)) * Log ODFV materialization skips instead of printing them ([feast-dev#7117](feast-dev#7117)) ([8fdcee7](feast-dev@8fdcee7)) * Pin build dependencies of source builds to the CI lock ([feast-dev#7104](feast-dev#7104)) ([dec2f24](feast-dev@dec2f24)) * Populate spec.project in registry REST feature view responses (#… ([feast-dev#7037](feast-dev#7037)) ([e304a84](feast-dev@e304a84)) * Preserve colliding source timestamps in Dask retrieval ([feast-dev#7071](feast-dev#7071)) ([1396c88](feast-dev@1396c88)) * Preserve Dask historical requests without eligible features ([feast-dev#7000](feast-dev#7000)) ([fff09f7](feast-dev@fff09f7)) * Read None elements of UUID and Decimal lists as None ([feast-dev#7039](feast-dev#7039)) ([03f30fe](feast-dev@03f30fe)) * Retire the support of Python3.10 ([feast-dev#7101](feast-dev#7101)) ([a56766e](feast-dev@a56766e)), closes [feast-dev#7104](feast-dev#7104) [feast-dev#7089](feast-dev#7089) [feast-dev#7106](feast-dev#7106) [feast-dev#7105](feast-dev#7105) * Serialize UUID and TimeUuid entity keys ([feast-dev#7032](feast-dev#7032)) ([0c845a1](feast-dev@0c845a1)) * Stop the Ray engine from hiding materialization errors ([feast-dev#7026](feast-dev#7026)) ([7903e2e](feast-dev@7903e2e)), closes [feast-dev#7024](feast-dev#7024) * **ui:** stop treating the monitoring baseline as the latest window ([feast-dev#7056](feast-dev#7056)) ([0f01802](feast-dev@0f01802)), closes [feast-dev#7054](feast-dev#7054) * Use date-only bounds for partition filters built from string timestamps ([feast-dev#7116](feast-dev#7116)) ([b1f2168](feast-dev@b1f2168)) * Validate entity column types in feast dbt import ([feast-dev#7067](feast-dev#7067)) ([abca80c](feast-dev@abca80c)), closes [feast-dev#5876](feast-dev#5876) * Warm up the whole DynamoDB connection pool ([feast-dev#6985](feast-dev#6985)) ([8ce548c](feast-dev@8ce548c)) ### Features * Add --gc-freeze option to feast serve ([feast-dev#7089](feast-dev#7089)) ([aad0c15](feast-dev@aad0c15)) * Add native complex types and scalar index tags to the Milvus online store ([feast-dev#7120](feast-dev#7120)) ([f7068d6](feast-dev@f7068d6)) * Add num_partitions and partition_key_isolation to the Milvus online store ([feast-dev#7083](feast-dev#7083)) ([68de84e](feast-dev@68de84e)), closes [feast-dev#6940](feast-dev#6940) * Add Pinecone online store integration ([feast-dev#6585](feast-dev#6585)) ([7a84f2a](feast-dev@7a84f2a)) * Add PolarisSource and PolarisProvider for Apache Polaris ([feast-dev#7006](feast-dev#7006)) ([386759c](feast-dev@386759c)) * Add row deletion to the Milvus online store ([feast-dev#7082](feast-dev#7082)) ([d3e2041](feast-dev@d3e2041)) * Add upstream FeatureView lineage support for PushSource ([feast-dev#6853](feast-dev#6853)) ([c9b9d72](feast-dev@c9b9d72)) * Optional nameless pre-computed vectors with a schema fingerprint ([feast-dev#7077](feast-dev#7077)) ([9420fac](feast-dev@9420fac)) ### Performance Improvements * Scan Lance sources lazily in the DuckDB offline store ([1299e4e](feast-dev@1299e4e)) Signed-off-by: Shuchu Han <[email protected]>

jiwidi requested a review from a team as a code owner October 28, 2026 11:05

This was referenced Oct 28, 2026

feat: Add num_partitions and partition_key_isolation to the Milvus online store #7083

Merged

fix: Break event timestamp ties by created timestamp in Milvus write batches #7084

Merged

test: Cover online_delete with an empty entity key list

cb241ef

Signed-off-by: jiwidi <[email protected]>

patelchaitany reviewed Oct 29, 2026

View reviewed changes

ci: Re-run the checks after a macOS runner timeout

408fc54

Signed-off-by: jiwidi <[email protected]>

patelchaitany approved these changes Nov 2, 2026

View reviewed changes

Merge branch 'master' into feat/milvus-online-delete

dd64a31

shuchu merged commit d3e2041 into feast-dev:master Nov 4, 2026
21 of 22 checks passed

This was referenced Nov 9, 2026

feat: Add a schema check and mutation timeout to the Milvus online store #7118

Merged

fix: Round-trip every Feast type and null in the Milvus online store #7119

Closed

feat: Add native complex types and scalar index tags to the Milvus online store #7120

Merged

patelchaitany mentioned this pull request Nov 11, 2026

chore: Sync upstream feast-dev/feast stable (0.68.0) opendatahub-io/feast#291

Merged

rhods-devops-app Bot mentioned this pull request Nov 12, 2026

Automated Sync from main to stable red-hat-data-services/feast#630

Open

Navigation Menu

Repository navigation

feat: Add row deletion to the Milvus online store - #7082

feat: Add row deletion to the Milvus online store#7082
shuchu merged 5 commits into
feast-dev:masterfrom
jiwidi:feat/milvus-online-delete

jiwidi commented Oct 28, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Oct 28, 2026 •

edited

Loading

Uh oh!

patelchaitany left a comment

Uh oh!

patelchaitany Oct 29, 2026

Uh oh!

patelchaitany Oct 29, 2026

Uh oh!

jiwidi commented Oct 29, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		collection = self._get_or_create_collection(config, table)
		self.client.delete(collection_name=collection["collection_name"], **selector)

Conversation

jiwidi commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Which issue(s) this PR fixes:

Checks

Testing Strategy

Misc

Uh oh!

codecov-commenter commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

patelchaitany left a comment

Choose a reason for hiding this comment

Uh oh!

patelchaitany Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

patelchaitany Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

jiwidi commented Oct 29, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jiwidi commented Oct 28, 2026 •

edited

Loading

codecov-commenter commented Oct 28, 2026 •

edited

Loading