You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 6991e76
Browse filesBrowse the repository at this point in the historyBrowse files
[fix](be) Score regexp and wildcard patterns with a constant like Elasticsearch (#69815)
### What problem does this PR solve?
Issue Number: None
Related PR: #69052
Problem Summary: score() of pattern queries disagrees with Elasticsearch
and with itself:
-- 'Inverted indexes are suitable for full text search',
-- 'The parser splits text into multiple terms', ...
SELECT score() FROM document WHERE SEARCH('content:/.*ext/'); -- BM25
sum
SELECT score() FROM document WHERE SEARCH('content:*ext'); -- BM25 sum
SELECT score() FROM document WHERE SEARCH('content:te*'); -- 1
SELECT score() FROM document WHERE content MATCH_REGEXP 'ext'; -- 0
Elasticsearch scores a query_string /regexp/, wildcard or prefix, and a
regexp, wildcard or prefix query, with a constant 1 per matching row
(default rewrite constant_score_blended); measured on 9.5.5 with the
rows above: 1.0 for both matching rows. Only an explicit
"rewrite": "scoring_boolean" sums the BM25 of the expanded terms. Next
to a scored clause in a bool must, a regexp adds exactly 1.
#69052 made SEARCH regexp and wildcard sum the BM25 of every expanded
term, i.e. scoring_boolean always on, while prefix stayed constant. That
sum also takes idf and avgdl from the current segment, not from the
tablet statistics MATCH uses: two inserts of the same text scored
0.2876821 and 1.3862942 for identical 'apple' rows, while MATCH 'apple'
scored 0.6931471 for both. MATCH_REGEXP never requested a scoring
context, so it scored 0 like a filter.
Fix:
1. Revert the scoring part of #69052: ExpandWeight scores every pattern
with the constant scorer again. The test fixture extensions that
#69051 built on are kept.
2. MATCH_REGEXP requests scoring on every index; the regexp expansion
then publishes the constant 1. The MATCH statistics collector skips
it, since a constant reads no statistics and the pattern is not text
to analyze.
0 commit comments