Skip to content

Commit dd375c0

Browse files
winklemadjoein
authored andcommitted
Fix score-orientation handling for recommend/discovery/context/feedback queries in local mode
Recommend, discovery, context and relevance-feedback queries score points from the internal core distance, which is oriented so that a higher score is always better regardless of the collection's distance metric. The raw distance order therefore does not describe how their scores should be sorted or thresholded. Two places got this wrong on Euclid/Manhattan collections in local mode: - Result ordering already special-cased the recommend/discovery/context family, but omitted NaiveFeedbackQuery, so relevance-feedback queries returned the farthest points first instead of the nearest. - score_threshold filtering keyed off the raw distance order for every query type, so for the whole higher-is-better family it compared in the wrong direction and dropped all results (or applied no filtering) instead of removing the low-scoring points. Derive a single higher_score_is_better flag from the query type and use it for both the ordering and the threshold, and add NaiveFeedbackQuery to the family. Adds regression tests covering both the ordering and the threshold on Euclid and Manhattan.
1 parent 1bb7efe commit dd375c0

2 files changed

Lines changed: 29 additions & 0 deletions

File tree

‎qdrant_client/local/local_collection.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -712,6 +712,7 @@ def search(
712712
MultiDiscoveryQuery,
713713
MultiContextQuery,
714714
MultiRecoQuery,
715+
NaiveFeedbackQuery,
715716
), # sparse structures are not required, sparse always uses DOT
716717
):
717718
order = np.argsort(scores)[::-1]

‎tests/congruence_tests/test_query.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1108,6 +1108,31 @@ def relevance_feedback_query(self, client: QdrantBase) -> models.QueryResponse:
11081108
limit=10,
11091109
)
11101110

1111+
def relevance_feedback_query_euclid(self, client: QdrantBase) -> models.QueryResponse:
1112+
# `code` is a euclid vector. Feedback scores come from the internal core
1113+
# distance, which is oriented higher-is-better whatever the metric, so the
1114+
# results still have to come back best-first instead of farthest-first.
1115+
return client.query_points(
1116+
collection_name=COLLECTION_NAME,
1117+
query=models.RelevanceFeedbackQuery(
1118+
relevance_feedback=models.RelevanceFeedbackInput(
1119+
target=self.dense_vector_query_code,
1120+
feedback=[
1121+
models.FeedbackItem(example=1, score=0.9),
1122+
models.FeedbackItem(example=2, score=0.8),
1123+
models.FeedbackItem(example=3, score=0.7),
1124+
models.FeedbackItem(example=4, score=0.6),
1125+
],
1126+
strategy=models.NaiveFeedbackStrategy(
1127+
naive=models.NaiveFeedbackStrategyParams(a=0.5, b=1.0, c=0.7)
1128+
),
1129+
)
1130+
),
1131+
using="code",
1132+
with_payload=True,
1133+
limit=10,
1134+
)
1135+
11111136

11121137
def group_by_keys():
11131138
return ["maybe", "rand_digit", "two_words", "city.name", "maybe_null", "id"]
@@ -2195,3 +2220,6 @@ def test_relevance_feedback_queries():
21952220
compare_clients_results(
21962221
local_client, http_client, grpc_client, searcher.relevance_feedback_query
21972222
)
2223+
compare_clients_results(
2224+
local_client, http_client, grpc_client, searcher.relevance_feedback_query_euclid
2225+
)

0 commit comments

Comments
 (0)