Skip to content

Multi-threaded vector search over multiple segments can lead to inconsistent results #14180

@benwtrent

Description

@benwtrent

Description

Related to: #14167

But multi-threaded search over multiple segments in addition to multi-leaf collection (e.g. information sharing) can lead to consistent results at low k values.

It is possible to get more consistent results, and likely remove most inconsistencies by simply gathering more neighbors (k in the query, fanout in lucene util, or efSearch if you will...). However, while HNSW search is approximate, we should strive for consistency.

Version and environment details

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions