Repository navigation
fix(vector-store): rank Euclidean results by distance in Qdrant and Pinecone - #2116
Conversation
…inecone Qdrant and Pinecone return a distance for Euclidean (and Qdrant's Manhattan): lower means closer. Both stores passed it through the x/(1+|x|) squash meant for unbounded Dot scores, which grows with the value, so the exact match got the lowest score and the farthest vector the highest. Read the collection or index metric and turn a distance d into 1 / (1 + d), as the SQLite store does. Cosine and Dot scores are unchanged, and so is any search where the metric can't be read. PineconeClient.create_index also sent metric "euclidean_distance", which Pinecone doesn't accept; it now sends "euclidean".
|
ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing |
PR Summary by QodoFix distance-based score direction in Qdrant and Pinecone
AI Description
Diagram
High-Level Assessment
Files changed (4)
|
Code Review by Qodo🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)
Great, no issues found!Qodo reviewed your code and found no material issues that require reviewTip of the day💡 Did you know, you can type 'qodo, fix this' on a finding and the fix lands right on your PR |
There was a problem hiding this comment.
LGTM.
Ran #2115 against a real local Qdrant: main returns [0.5, 0.75, 0.875], this branch returns [1.0, 0.5, 0.25]. The inversion was real, and 1/(1+d) matches what the sqlite backend already does, so the contract holds across backends now.
Also confirmed the metric reaches scoring — PineconeStore.search_vectors goes through PineconeSearch to PineconeIndex.search_vectors, and Distance.EUCLID == "Euclid" is true for the enum, so the Qdrant branch isn't dead code.
Reverting both files to main takes 10 tests down. Separately, euclidean_distance -> euclidean fixes an index that could never have been created, worth one line in the description so it shows up in the changelog. Two more notes in the comment below, neither blocking.
|
Two small things from my side, neither blocking:
Related: when the metric read fails we fall back to the old formula. On a euclidean index that flips the ranking back, and only the log line says so. Might be worth making that louder than a warning. Also |
Description
For Euclidean, Qdrant and Pinecone return a distance: lower means closer. Both stores passed it through
(x / (1 + |x|) + 1) / 2, the squash added for unbounded Dot scores, and that grows withx. So the exact match got the lowest score and the farthest vector the highest, against the "higher is better" contract invector_store.py.Both stores now read the metric of the collection or index and turn a distance
dinto1 / (1 + d), assqlite_vec_store.pydoes. With the example from #2115, the scores go from[0.5, 0.75, 0.875]to[1.0, 0.5, 0.25]. Cosine and Dot scores don't change.Type of Change
Related Issues
Fixes #2115
Changes Made
semantica/vector_store/qdrant_store.py:QdrantCollection.search_points()reads the collection's vector config once (get_collection), so it also works for a collection it attached to or reopened from disk. For named vectors it uses the config of the vector inusing. Euclid and Manhattan distances become1 / (1 + d).semantica/vector_store/pinecone_store.py:PineconeIndextakes the index metric, whichPineconeStorereads withdescribe_indexwhen it creates or attaches an index.euclideanscores become1 / (1 + d).create_index(metric="euclidean")now sends"euclidean"instead of"euclidean_distance", which isn't one of Pinecone's metrics (cosine,euclidean,dotproduct), so a Euclidean index couldn't be created at all.tests/vector_store/test_qdrant_score_direction.pyruns real Qdrant local mode, for a created and a reopened collection, a collection created outside Semantica (Euclid and Manhattan, plain and named vectors), and pins the current Cosine and Dot scores.tests/vector_store/test_pinecone_score_direction.pydoes the same with a fake Pinecone client, plus the metric name sent on create.Testing
python -m build)The Euclidean, Manhattan and metric-name cases fail on
mainand pass with this change. The Cosine and Dot cases pass on both.pytest tests/vector_storepasses (withouttest_performance_benchmarks.py, which needspsutil). black, isort and flake8 at the pre-commit versions are clean on the changed lines and the new tests. I haven't run it against a live Pinecone index.Documentation
Breaking Changes
Breaking Changes: No for Cosine and Dot. Euclidean and Manhattan scores change from the old increasing values to
1 / (1 + d), so a threshold someone tuned on the old scores needs a new value.Checklist