Skip to content

FAISS backends ignore the requested distance metric #89

Open

Description

@Marius1311

The faiss-cpu and faiss-gpu backends accept metric but always build IndexFlatL2, so e.g. knn_dist_metric="cosine" silently returns Euclidean neighbors.

  • class _FaissCpuBackend(_KNNBackend):
    def __init__(
    self,
    n_neighbors: int,
    metric: str,
    random_state: int = 0,
    **kwargs: Any,
    ):
    check_deps("faiss-cpu")
    import faiss
    self.faiss = faiss
    self._index = None
    def fit(self, data: np.ndarray) -> None:
    dims = data.shape[1]
    index = self.faiss.IndexFlatL2(dims)
    # Ensure data is float32 and C-contiguous
    data_f32 = np.ascontiguousarray(data.astype(np.float32))
    index.add(data_f32)
    self._index = index
    def query(self, points: np.ndarray, k: int) -> tuple[np.ndarray, np.ndarray]:
    # Ensure points are float32 and C-contiguous
    points_f32 = np.ascontiguousarray(points.astype(np.float32))
  • class _FaissGpuBackend(_KNNBackend):
    def __init__(
    self,
    n_neighbors: int,
    metric: str,
    random_state: int = 0,
    **kwargs: Any,
    ):
    check_deps("faiss-gpu")
    import faiss
    self.faiss = faiss
    self.res = faiss.StandardGpuResources()
    self._index = None
    def fit(self, data: np.ndarray) -> None:
    dims = data.shape[1]
    flat = self.faiss.IndexFlatL2(dims)
    gpu_index = self.faiss.index_cpu_to_gpu(self.res, 0, flat)
    # Ensure data is float32 and C-contiguous

Expected: honour the metric (e.g. IndexFlatIP on L2-normalised data for cosine) or raise for unsupported metrics. The only FAISS test checks importability (tests/model/test_kernel.py), so a test comparing neighbors against the sklearn backend would catch this.

Related: squared distances from FAISS (separate issue).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions