You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 9c71f1f
Browse filesBrowse the repository at this point in the historyBrowse files
[fix](nereids) Keep ANN TopN and bind score filters to one scan
### What problem does this PR solve?
Issue Number: None
Related PR: #69292
Problem Summary:
Follow-up fixes for the standalone minimum score filter (score() > x without
a score TopN):
1. Binding the score range replaced the scan's ANN TopN with an empty one, so
WHERE content MATCH 'x' AND score() > 0.5 ORDER BY l2_distance_approximate()
LIMIT k no longer used the ANN index and computed every distance by brute
force. The scan applies the score range before the ANN search, so the
binding now keeps the ANN order keys and limit. On a local cluster the
profile of this query shows AnnIndexTopNSearchCnt=1 and no brute-force
fallback, and the nearest row that fails the score filter is not returned.
2. score() has no input slots, so join rules treated score() > x like a
constant. PushDownFilterThroughJoin copied it into both children,
PushFilterInsideJoin turned it into a join condition and
PushDownJoinOtherCondition then pushed it into the left child. A query with
an IN or EXISTS subquery failed with a misleading "must contain MATCH"
error, and an inner join silently filtered whichever table was on the left.
The predicate now goes only into the child whose rows all reach the output
of a left/right outer, semi or anti join, as ReorderJoin already does.
Inner, cross and full joins keep it above the join, where CheckScoreUsage
rejects it with a message that suggests a subquery; score() in a JOIN
condition fails analysis.
3. test_bm25_score still expected an error for a score() > 0.5 filter without
ORDER BY and LIMIT, which is now supported. It now checks that the row count
matches the score TopN path.
4. EXPLAIN prints the pushed range (SCORE RANGE: score() > 0.5), because the
predicate is no longer among the scan conjuncts.
5. The new filter rules are guarded by the cached containsType check, so
filters without score() skip them.
6. SearchExpression.toSql keeps the options argument of search(dsl, options).
### Release note
score() filters work with IN and EXISTS subqueries and on the preserved side of
left or right outer joins. With inner, cross or full joins, or in a JOIN
condition, they fail with a clear error instead of filtering an arbitrary
table. MATCH queries with a score() filter and an ANN TopN keep using the ANN
index. EXPLAIN shows the pushed score range.
### Check List (For Author)
- Test: Unit Test and Regression test
- FE unit tests: CheckScoreUsageTest, PushDownFilterThroughJoinTest,
PushFilterInsideJoinTest, SearchExpressionTest, RewriteSearchToSlotsTest,
PullUpProjectExprUnderTopNTest, CheckAnalysisTest, ReorderJoinTest
(180 tests).
- Regression tests on a local ASAN cluster: test_bm25_score,
test_score_filter_usage, test_score_filter_matrix,
test_bm25_score_range_filter, test_search_score_topn_predicates and six
more score suites in inverted_index_p0, all 36 suites in search,
ann_with_fulltext and ann_topn_small_candidate_fallback.
- Behavior changed: Yes. score() filters above inner, cross or full joins and
in JOIN conditions are rejected; outer, semi and anti joins keep the filter
on their output side; EXPLAIN shows SCORE RANGE.
- Does this need documentation: Yes, with the score() documentation for
standalone minimum score filters.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
0 commit comments