Skip to content

Commit cc615ab

Browse files
committed
[fix](be) Merge master and migrate search scoring to unified indexes
Problem Summary: Resolve conflicts with the unified MATCH and SEARCH index execution refactor. Preserve this PR's scoring behavior through the new query and reader interfaces. Related PR: #68758 Release note: Preserve search scoring fixes after index execution unification. Tests: ASAN unit validation in progress; header hygiene and clang-format 16 passed. Behavior changed: No additional behavior beyond the existing scoring fix. Documentation: Not required; no new syntax or settings.
2 parents be4b4e8 + 3b8eb5b commit cc615ab

1,157 files changed

Lines changed: 46181 additions & 38542 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.clang-format-ignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,5 +8,5 @@ be/src/util/sse2neo.h
88
be/src/util/sse2neon.h
99
be/src/util/mustache/mustache.h
1010
be/src/util/mustache/mustache.cc
11-
be/src/storage/index/inverted/analyzer/kuromoji/dict/darts.h
11+
be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/darts.h
1212
cloud/src/common/defer.h

‎.github/CODEOWNERS‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,7 @@ fe/fe-core/src/main/java/org/apache/doris/fs @CalvinKirs @924060929
2525
fe/fe-core/src/main/java/org/apache/doris/fsv2 @CalvinKirs
2626
be/src/olap/rowset/segment_v2/variant @eldenmoon @csun5285
2727
be/src/storage/index/inverted/ @airborne12 @eldenmoon @csun5285
28-
be/src/exprs/function/function_search.h @airborne12 @eldenmoon @csun5285
29-
be/src/exprs/function/function_search.cpp @airborne12 @eldenmoon @csun5285
28+
be/src/exprs/function/search/ @airborne12 @eldenmoon @csun5285
3029
be/src/exprs/function/match.cpp @airborne12 @eldenmoon @csun5285
3130
be/src/exprs/function/match.h @airborne12 @eldenmoon @csun5285
3231
be/src/exprs/vsearch.h @airborne12 @eldenmoon @csun5285

‎.github/workflows/comment-to-trigger-teamcity.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,6 +138,10 @@ jobs:
138138
echo "COMMENT_USER_ID ${COMMENT_USER_ID} is allowed to skip buildall for branch-4.1"
139139
echo "comment_trigger=false" | tee -a "$GITHUB_OUTPUT"
140140
echo "comment_skip=true" | tee -a "$GITHUB_OUTPUT"
141+
elif [[ "${COMMENT_USER_ID}" == '9208457' && "${TARGET_BRANCH}" == 'branch-4.2' ]]; then
142+
echo "COMMENT_USER_ID ${COMMENT_USER_ID} is allowed to skip buildall for branch-4.2"
143+
echo "comment_trigger=false" | tee -a "$GITHUB_OUTPUT"
144+
echo "comment_skip=true" | tee -a "$GITHUB_OUTPUT"
141145
else
142146
echo "COMMENT_USER_ID ${COMMENT_USER_ID} is not allowed to skip buildall."
143147
exit

‎.licenserc.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ header:
126126
- "be/src/util/mustache/mustache.h"
127127
- "be/src/util/sse2neo.h"
128128
- "be/src/util/sse2neon.h"
129-
- "be/src/storage/index/inverted/analyzer/kuromoji/dict/darts.h"
129+
- "be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/darts.h"
130130
- "be/src/pch/*"
131131
- "be/test/data"
132132
- "be/test/expected_result"

‎LICENSE.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -369,7 +369,7 @@ be/src/olap/skiplist.h : BSD-style license
369369

370370
--------------------------------------------------------------------------------
371371

372-
be/src/storage/index/inverted/analyzer/kuromoji/dict/darts.h : 2-Clause BSD license (darts-clone)
372+
be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/darts.h : 2-Clause BSD license (darts-clone)
373373

374374
Copyright (c) 2008-2014 Susumu Yata. All rights reserved.
375375
Vendored from https://github.com/s-yata/darts-clone

‎NOTICE.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ This software includes third party software subject to the following copyrights:
7575
- Jackson-dataformat-cbor - https://github.com/FasterXML/jackson-dataformats-binary
7676
- Darts-clone (double-array trie) - Copyright 2008-2014 Susumu Yata - https://github.com/s-yata/darts-clone (BSD 2-clause)
7777
- mecab-ipadic (IPADIC) Japanese morphological dictionary - Copyright 2000-2003 Nara Institute of Science and Technology (NAIST) - licensed under NAIST-2003 (BSD-style); the kuromoji analyzer bundles the UTF-8 form from https://github.com/lindera/mecab-ipadic (content of mecab-ipadic-2.7.0-20070801).
78-
- Apache Lucene - https://github.com/apache/lucene (Apache-2.0): the kuromoji Japanese analyzer under be/src/storage/index/inverted/analyzer/kuromoji is an independent C++ implementation modeled on Lucene's kuromoji analyzer (JapaneseTokenizer), including its search-mode compound-decomposition cost model.
78+
- Apache Lucene - https://github.com/apache/lucene (Apache-2.0): the kuromoji Japanese analyzer under be/src/storage/index/inverted/analysis/analyzer/kuromoji is an independent C++ implementation modeled on Lucene's kuromoji analyzer (JapaneseTokenizer), including its search-mode compound-decomposition cost model.
7979

8080
The licenses for these third party components are included in LICENSE.txt
8181

‎be/benchmark/benchmark_gram_extractor.hpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
#include <string_view>
2525
#include <vector>
2626

27-
#include "storage/index/inverted/gram/gram_extractor.h"
27+
#include "storage/index/inverted/analysis/gram/gram_extractor.h"
2828

2929
namespace doris::segment_v2::gram::benchmark_detail {
3030

‎be/src/cloud/cloud_schema_change_job.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333
#include "common/status.h"
3434
#include "service/backend_options.h"
3535
#include "storage/delete/delete_handler.h"
36-
#include "storage/index/inverted/inverted_index_desc.h"
36+
#include "storage/index/inverted/common/inverted_index_desc.h"
3737
#include "storage/olap_define.h"
3838
#include "storage/rowset/beta_rowset.h"
3939
#include "storage/rowset/rowset.h"

‎be/src/cloud/cloud_tablet.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@
5151
#include "io/cache/block_file_cache_factory.h"
5252
#include "storage/compaction/compaction.h"
5353
#include "storage/compaction/cumulative_compaction_time_series_policy.h"
54-
#include "storage/index/inverted/inverted_index_desc.h"
54+
#include "storage/index/inverted/common/inverted_index_desc.h"
5555
#include "storage/olap_define.h"
5656
#include "storage/rowset/beta_rowset.h"
5757
#include "storage/rowset/rowset.h"

‎be/src/cloud/cloud_warm_up_manager.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@
4646
#include "runtime/cluster_info.h"
4747
#include "runtime/exec_env.h"
4848
#include "service/backend_options.h"
49-
#include "storage/index/inverted/inverted_index_desc.h"
49+
#include "storage/index/inverted/common/inverted_index_desc.h"
5050
#include "storage/rowset/beta_rowset.h"
5151
#include "storage/tablet/tablet.h"
5252
#include "util/brpc_client_cache.h" // BrpcClientCache

0 commit comments

Comments
 (0)