Skip to content

branch-4.1: [fix](nereids) Do not infer MATCH and SEARCH predicates across column equalities #69419 - #69564

Merged
yiguolei merged 2 commits into
apache:branch-4.1from
airborne12:backport/69419-branch-4.1
Oct 26, 2026
Merged

yiguolei merged 2 commits into
apache:branch-4.1from
airborne12:backport/69419-branch-4.1

Conversation

@airborne12

@airborne12 airborne12 commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

What problem does this PR solve?

Related PR: #69419

Problem Summary: Backport #69419 to branch-4.1.

Predicate inference copied a predicate on one column to every column that is equal to it, also when the predicate contained MATCH, multi_match or SEARCH. Their result depends on the inverted index of the column, not only on its value: with c1 = c2 and different parsers on the two columns, the copy removed matching rows, and with the same index it doubled score(), so a minimum score let low-scored rows through. InferPredicateByReplace.infer now skips every input that contains Match, MultiMatch, Search or SearchExpression.

Branch adaptations

The branch does not have the NoneMovableFunction check that master has next to the new check, so only the new check is added. The test imports SearchDslParser from its package on the branch. infer_match_predicate sets enable_common_expr_pushdown = true, as the score suites of the branch do: CI on branch-4.1 draws it at random for each session, and without it MATCH is not evaluated with the index in the scan and score() is 0.

Release note

Fix wrong results and scores when predicate inference copied a MATCH, multi_match or SEARCH predicate to an equal column.

Check List (For Author)

  • Test
    • Regression test
    • Unit Test
    • Manual test
  • Behavior changed:
    • Yes. Predicate inference no longer copies a predicate that contains MATCH, multi_match or SEARCH to an equal column.
  • Does this need documentation?
    • No.

Validation

Validated together with the other two backports (#69399, #69419 and #69292 on branch-4.1 at 3f911d0), in a local ASAN build:

  • PASS: FE unit tests LogicalPlanDeepCopierTest, CTEInlineTest, InferPredicateByReplaceTest, InferPredicatesTest, UnequalPredicateInferTest, BindScoreToOlapScanTest, SearchExpressionTest, TopNRuntimeFilterTest and MaterializedViewUtilsTest (228 tests), and FE Checkstyle.
  • PASS: ./build.sh --be and ./build.sh --fe.
  • PASS: BE unit tests SegmentIteratorScoreCommonExprTest, SegmentIteratorApplyIndexExprTest and VirtualSlotRefTest (31 tests); clang-format 16 on the changed BE files.
  • PASS: the 16 regression suites of the branch that use score(), including test_cte_inline_virtual_column, infer_match_predicate, test_score_filter_usage and test_score_filter_matrix. The framework regenerated the .out of test_score_filter_matrix on this branch: it drops the 46 SNII cases and keeps the 94 V2 and V3 cases of master unchanged. The other .out files are unchanged.
  • PASS: with the fuzzy session variables of CI (use_fuzzy_session_variable = true and its pull request number), the score suites failed in sessions that drew enable_common_expr_pushdown = false; with the setting pinned, four runs of test_score_filter_usage, test_score_filter_matrix and test_cte_inline_virtual_column, and four of infer_match_predicate, passed.

🤖 Generated with Claude Code

… equalities

### What problem does this PR solve?

Related PR: apache#69419

Problem Summary: Backport apache#69419 to branch-4.1. Predicate inference
copied a predicate on one column to every column that is equal to it,
also when the predicate contained MATCH, multi_match or SEARCH. Their
result depends on the inverted index of the column, not only on its
value: with c1 = c2 and different parsers on the two columns, the copy
removed matching rows, and with the same index it doubled score().
InferPredicateByReplace.infer now skips every input that contains Match,
MultiMatch, Search or SearchExpression.

The branch does not have the NoneMovableFunction check that master has
next to the new check, so only the new check is added. The test imports
SearchDslParser from its package on the branch.

### Release note

Fix wrong results and scores when predicate inference copied a MATCH,
multi_match or SEARCH predicate to an equal column.

### Check List (For Author)

- Test: Includes the unit and regression tests of apache#69419; results are
  recorded in the pull request.
- Behavior changed: Yes. Predicate inference no longer copies a predicate
  that contains MATCH, multi_match or SEARCH to an equal column.
- Does this need documentation: No.

Source-PR: apache#69419
Source-Head: 9e661c8
Co-Authored-By: Claude Opus 5.5 <[email protected]>
@airborne12
airborne12 requested a review from yiguolei as a code owner October 24, 2026 05:16
@airborne12

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

### What problem does this PR solve?

Related PR: apache#69419

Problem Summary: CI on branch-4.1 draws enable_common_expr_pushdown
at random for each session. With it off, MATCH is not evaluated
with the inverted index in the scan, so no BM25 score is
collected and score() returns 0, which fails the score cases of
infer_match_predicate. Like the score suites of the branch, the suite
sets enable_common_expr_pushdown = true.

### Release note

None

### Check List (For Author)

- Test: Regression test. With use_fuzzy_session_variable = true and
the pull request number of CI, the score query of the suite returned
score 0 in the 10 of 24 sessions that drew enable_common_expr_pushdown
= false; with the setting pinned, four runs of the suite passed.
- Behavior changed: No
- Does this need documentation: No

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@airborne12

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@airborne12

Copy link
Copy Markdown
Member Author

run nonConcurrent

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@yiguolei
yiguolei merged commit 5b87c1b into apache:branch-4.1 Oct 26, 2026
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants