Skip to content

Commit a8cc117

Browse files
committed
feat: added milvus multi tenancy on database&collection-level(feast-dev#5652)
1 parent b70eda0 commit a8cc117

3 files changed

Lines changed: 680 additions & 18 deletions

File tree

‎docs/reference/online-stores/milvus.md‎

Lines changed: 99 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,23 @@
1-
# Redis online store
1+
# Milvus online store
22

33
## Description
44

55
The [Milvus](https://milvus.io/) online store provides support for materializing feature values into Milvus.
66

77
* The data model used to store feature values in Milvus is described in more detail [here](../../specs/online\_store\_format.md).
8+
* Multi-tenancy behavior is controlled by the `multi_tenancy_mode` setting (see below).
89

910
## Getting started
1011
In order to use this online store, you'll need to install the Milvus extra (along with the dependency needed for the offline store of choice). E.g.
1112

1213
`pip install 'feast[milvus]'`
1314

14-
You can get started by using any of the other templates (e.g. `feast init -t gcp` or `feast init -t snowflake` or `feast init -t aws`), and then swapping in Redis as the online store as seen below in the examples.
15+
You can get started by using any of the other templates (e.g. `feast init -t gcp` or `feast init -t snowflake` or `feast init -t aws`), and then swapping in Milvus as the online store as seen below in the examples.
1516

1617
## Examples
1718

19+
### Local mode (file-based)
20+
1821
Connecting to a local MilvusDB instance:
1922

2023
{% code title="feature_store.yaml" %}
@@ -25,7 +28,25 @@ provider: local
2528
online_store:
2629
type: milvus
2730
path: "data/online_store.db"
28-
connection_string: "localhost:6379"
31+
embedding_dim: 128
32+
index_type: "FLAT"
33+
metric_type: "COSINE"
34+
```
35+
{% endcode %}
36+
37+
### Remote mode
38+
39+
Connecting to a remote Milvus instance:
40+
41+
{% code title="feature_store.yaml" %}
42+
```yaml
43+
project: my_feature_repo
44+
registry: data/registry.db
45+
provider: local
46+
online_store:
47+
type: milvus
48+
host: "http://localhost"
49+
port: 19530
2950
embedding_dim: 128
3051
index_type: "FLAT"
3152
metric_type: "COSINE"
@@ -34,6 +55,80 @@ online_store:
3455
```
3556
{% endcode %}
3657
58+
### Multi-Tenancy Modes
59+
60+
Feast supports three mutually exclusive multi-tenancy strategies for Milvus. Set `multi_tenancy_mode` to make intent explicit; legacy configs are inferred for backwards compatibility.
61+
62+
- `none` (default): single-tenant; disallows `db_name` and `collection_name_prefix`
63+
- `database`: per-tenant database isolation; requires `db_name`; forbids `collection_name_prefix`; not supported with local (Milvus Lite) mode
64+
- `collection`: shared database with per-tenant collection prefix; requires `collection_name_prefix`; forbids `db_name`
65+
66+
#### Database isolation (`multi_tenancy_mode: database`)
67+
68+
For multi-tenant environments, you can use database-level isolation by specifying the `db_name` parameter. This creates a separate Milvus database for each tenant, providing logical isolation of collections and data:
69+
70+
{% code title="feature_store.yaml" %}
71+
```yaml
72+
# Marketing team configuration
73+
project: marketing_features
74+
registry: data/registry.db
75+
provider: remote
76+
online_store:
77+
type: milvus
78+
host: "http://localhost"
79+
port: 19530
80+
multi_tenancy_mode: database
81+
db_name: "marketing_db"
82+
embedding_dim: 768
83+
index_type: "IVF_FLAT"
84+
metric_type: "COSINE"
85+
```
86+
{% endcode %}
87+
88+
{% code title="feature_store.yaml" %}
89+
```yaml
90+
# Sales team configuration
91+
project: sales_features
92+
registry: data/registry.db
93+
provider: remote
94+
online_store:
95+
type: milvus
96+
host: "http://localhost"
97+
port: 19530
98+
multi_tenancy_mode: database
99+
db_name: "sales_db"
100+
embedding_dim: 128
101+
index_type: "FLAT"
102+
metric_type: "L2"
103+
```
104+
{% endcode %}
105+
106+
With database-level multi-tenancy:
107+
- Each team's collections are isolated in their own database
108+
- Different embedding dimensions can be used per database
109+
- Different vector index types and metrics can be configured per team
110+
- Access control can be managed at the database level in Milvus
111+
112+
#### Collection prefix isolation (`multi_tenancy_mode: collection`)
113+
114+
Use a shared Milvus database and isolate tenants via collection name prefixes:
115+
116+
{% code title="feature_store.yaml" %}
117+
```yaml
118+
project: shared_features
119+
registry: data/registry.db
120+
provider: remote
121+
online_store:
122+
type: milvus
123+
host: "http://localhost"
124+
port: 19530
125+
multi_tenancy_mode: collection
126+
collection_name_prefix: "tenant_alpha"
127+
embedding_dim: 128
128+
index_type: "FLAT"
129+
metric_type: "COSINE"
130+
```
131+
{% endcode %}
37132

38133
The full set of configuration options is available in [MilvusOnlineStoreConfig](https://rtd.feast.dev/en/latest/#feast.infra.online_stores.milvus.MilvusOnlineStoreConfig).
39134

@@ -61,5 +156,6 @@ Below is a matrix indicating which functionality is supported by the Milvus onli
61156
| collocated by feature service | no |
62157
| collocated by entity key | no |
63158
| vector similarity search | yes |
159+
| database-level multi-tenancy | yes |
64160

65161
To compare this set of functionality against other online stores, please see the full [functionality matrix](overview.md#functionality-matrix).

0 commit comments

Comments
 (0)