make vector search configs part of feature definition

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Hello everyone,
I have a question about the recommended pattern for using Milvus as an online store in a multi-team environment.
The feature_store.yaml configuration for Milvus defines global configs, like:
embedding_dim
index_type
metric_type
Our use case involves multiple teams (e.g., marketing, sales) sharing the feature store, but their vector embeddings have different requirements. For example:
The Marketing team might use a 768-dimension vector with COSINE similarity.
The Sales team might use a 128-dimension vector with L2 similarity.
Since these configurations seem to be set at the online_store level, it implies we can only support one type of vector across the entire store. This would prevent us from sharing one feature server across teams with different vector models.
What is the best way to support multiple, diverse vector feature views (with different dimensions, metrics, etc.) in a single Milvus online store?
Is the intended pattern to create a separate online_store definition for each unique vector configuration, or am I misunderstanding how these parameters should be used?
Thanks for any insights!

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

The Milvus support four different patterns of multi-tenancy: https://milvus.io/docs/multi_tenancy.md
Maybe we can suppor the following two:
1, database-level
2, collection-level

feat: added milvus multi tenancy on database&collection-level(feast-d…

Suggesting this issue's scope should cover the index build parameters, not only the search configs, and that they should be typed rather than a dict. Two pieces of evidence from working with vector indexes recently.

1. `metric` is half-migrated; `index_type` and the build params are not

The Milvus store already honours a per-field metric but takes everything else from global config — milvus.py:409-410:

metric_type=metric or config.online_store.metric_type,   # Field can override
index_type=config.online_store.index_type,               # global only

So the split today is:

parameter	source
`vector_search_metric`	`Field`, falling back to global
`index_type`	global `feature_store.yaml` only
`nlist`, `M`, `efConstruction`, …	global only, inside an untyped dict

That last row is milvus.py:227-229:

# Index build params for vector fields, e.g. {"M": 16, "efConstruction": 200}.
index_params: Optional[Dict[str, Any]] = None

Which means the multi-tenant problem this issue opens with applies to build parameters at least as strongly as to embedding_dim: two feature views needing different M or nlist cannot coexist, and Field has nowhere to put them — it carries only vector_index, vector_length and vector_search_metric (field.py:48-50).

2. An untyped dict makes a whole class of error invisible

Worth arguing for typed fields rather than extending Dict[str, Any], because unknown keys are silently dropped by the engines underneath.

Measured against pylance 12.0.0 — create_index takes HNSW parameters through **kwargs and forwards them to Rust, where the names are m / ef_construction. The Java/Scala binding for the same library spells them M / efConstruction. Passing the Java spellings:

m=16,  ef_construction=200   ->  index_stats: m=16  ef_construction=200   correct
M=16,  efConstruction=200    ->  index_stats: m=20  ef_construction=150   silently defaulted
totally_bogus_param=123      ->  index_stats: m=20  ef_construction=150   no error raised

The build succeeds and reports success while producing a default-parameter index. The only way to tell is to read index_stats back afterwards. Lance's defaults here happen not to match the values that were asked for, so this is not a theoretical difference.

A Dict[str, Any] on the config object cannot catch that; a typed field on Field can, at feast apply time.

3. Why this matters beyond multi-tenancy

Reviewing an unrelated production index pipeline recently, its HNSW build parameters were encoded in the name of the build step — the tuple efConstruction=2000, ef=500, M=20 appeared only as a substring of a job name, with nothing in any schema or registry. Nothing recorded which parameters produced which artifact, and recall was never measured, so a parameter change was indistinguishable from a data change after the fact.

That is the failure mode worth designing against: if build parameters live anywhere other than the versioned feature definition, there is no object to attach a measurement to. "This version scored recall@10 = 0.95" is only meaningful if "this version" also pins the parameters that produced it.

So concretely, in addition to what this issue already proposes:

typed build-parameter fields on Field (or a small typed object it holds), covering at least index_type and the per-type build parameters
Field carrying them means they are in the registry and move with the feature view version
engines read them from the feature definition, with global config as fallback for compatibility

Happy to take the Field-side change if the direction looks right. The engine-side plumbing is larger and probably wants splitting per store.

Navigation Menu

Repository navigation

make vector search configs part of feature definition #5652

1. `metric` is half-migrated; `index_type` and the build params are not

2. An untyped dict makes a whole class of error invisible

3. Why this matters beyond multi-tenancy

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

make vector search configs part of feature definition #5652

Description

Activity

shuchu commented on Nov 2, 2025

haoxu0 commented on Oct 5, 2026

1. metric is half-migrated; index_type and the build params are not

2. An untyped dict makes a whole class of error invisible

3. Why this matters beyond multi-tenancy

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

1. `metric` is half-migrated; `index_type` and the build params are not