Skip to content

Commit c54b0b6

Browse files
csun5285claude
andcommitted
[fix](be) Do not push storage TopN to a MOR table read as DUP
### What problem does this PR solve? Issue Number: None Related PR: #62222 Problem Summary: A merge-on-read UNIQUE table listed in read_mor_as_dup_tables is read without merging, so OlapScanLocalState::_storage_no_merge() is true for it. Since #62222 the scanner alone decides whether VCollectIterator takes the storage TopN path, and it pushes read_orderby_key_limit for every key TopN scan where _storage_no_merge() holds. TabletReader::_init_orderby_keys_param() prepares the key columns that VCollectIterator::_topn_next() compares only for DUP_KEYS and merge-on-write UNIQUE_KEYS tablets, so for a MOR table read as DUP read_orderby_key_columns stays null and the DORIS_CHECK at the top of _topn_next() aborts the BE. FE pushes the sort info to the scan for any table whose ORDER BY is a key prefix, so a plain query is enough: -- mor_tbl: UNIQUE KEY(k1, k2), enable_unique_key_merge_on_write = false SET read_mor_as_dup_tables = 'mor_tbl'; SELECT * FROM mor_tbl ORDER BY k1, k2 LIMIT 4; -- ERROR 1105 (HY000): RpcException, msg: send fragments failed. -- be.out: Check failed: _reader->_reader_context.read_orderby_key_columns != nullptr The scanner now keeps segment LIMIT off for a key TopN on a MOR table read as DUP. The scan returns every row and the TopN operator above sorts them, the same as any key TopN that is not pushed to storage. A plain LIMIT on such a table is still pushed to SegmentIterator. ### Release note Fix a BE crash on ORDER BY <key> LIMIT queries over a merge-on-read unique table listed in read_mor_as_dup_tables. ### Check List (For Author) - Test: Regression test - unique_with_mor_p0/test_read_mor_as_dup_key_topn: aborts the BE before this change, passes after. - unique_with_mor_p0/test_read_mor_as_dup and all 19 suites in query_p0/limit pass. - Behavior changed: No - Does this need documentation: No Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
1 parent 038e227 commit c54b0b6

3 files changed

Lines changed: 96 additions & 3 deletions

File tree

‎be/src/exec/scan/olap_scanner.cpp‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -475,6 +475,9 @@ Status OlapScanner::_init_tablet_reader_params(
475475
const bool no_runtime_filters = _total_rf_num == 0;
476476
const bool segment_limit_enabled = _state->enable_segment_limit_pushdown();
477477
const bool storage_no_merge = olap_scan_local_state->_storage_no_merge();
478+
// Storage TopN compares key columns that TabletReader prepares only for DUP and
479+
// merge-on-write tablets, so a MOR table read as DUP cannot use it.
480+
const bool storage_topn_supported = storage_no_merge && !read_mor_as_dup;
478481

479482
if (_limit > 0 && no_runtime_filters && segment_limit_enabled && storage_no_merge) {
480483
for (const auto& conjunct : _conjuncts) {
@@ -486,9 +489,10 @@ Status OlapScanner::_init_tablet_reader_params(
486489

487490
// Segment LIMIT has only two legal states: completely disabled, or enabled after every
488491
// row-filtering conjunct has become a storage predicate or SegmentIterator common expr.
489-
const bool can_push_down_segment_limit = _limit > 0 && no_runtime_filters &&
490-
_conjuncts.empty() && segment_limit_enabled &&
491-
storage_no_merge;
492+
// A key TopN also needs storage TopN support, otherwise nothing is pushed.
493+
const bool can_push_down_segment_limit =
494+
_limit > 0 && no_runtime_filters && _conjuncts.empty() && segment_limit_enabled &&
495+
storage_no_merge && (!has_key_topn || storage_topn_supported);
492496
if (can_push_down_segment_limit) {
493497
if (has_key_topn) {
494498
_tablet_reader_params.read_orderby_key = true;
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
-- This file is automatically generated. You should know what you did if you want to edit this
2+
-- !merged --
3+
1 1 12
4+
2 2 20
5+
3 3 30
6+
7+
-- !dup_asc --
8+
1 1 10
9+
1 1 11
10+
1 1 12
11+
2 2 20
12+
13+
-- !dup_desc --
14+
7 7 70
15+
6 6 60
16+
5 5 50
17+
18+
-- !dup_prefix --
19+
1 1 10
20+
1 1 11
21+
1 1 12
22+
2 2 20
23+
24+
-- !dup_filter --
25+
1 1 11
26+
1 1 12
27+
2 2 20
28+
29+
-- !dup_offset --
30+
2 2 20
31+
3 3 30
32+
33+
-- !dup_limit --
34+
5
35+
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
// Licensed to the Apache Software Foundation (ASF) under one
2+
// or more contributor license agreements. See the NOTICE file
3+
// distributed with this work for additional information
4+
// regarding copyright ownership. The ASF licenses this file
5+
// to you under the Apache License, Version 2.0 (the
6+
// "License"); you may not use this file except in compliance
7+
// with the License. You may obtain a copy of the License at
8+
//
9+
// http://www.apache.org/licenses/LICENSE-2.0
10+
//
11+
// Unless required by applicable law or agreed to in writing,
12+
// software distributed under the License is distributed on an
13+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
// KIND, either express or implied. See the License for the
15+
// specific language governing permissions and limitations
16+
// under the License.
17+
18+
// ORDER BY a key prefix with LIMIT on a MOR table listed in read_mor_as_dup_tables.
19+
// The table is read as DUP, so every version of a key is a row and the TopN sees all of them.
20+
suite("test_read_mor_as_dup_key_topn") {
21+
sql "DROP TABLE IF EXISTS test_read_mor_as_dup_key_topn"
22+
sql """
23+
CREATE TABLE test_read_mor_as_dup_key_topn (
24+
k1 INT NOT NULL,
25+
k2 INT NOT NULL,
26+
v INT NULL
27+
) ENGINE=OLAP
28+
UNIQUE KEY(k1, k2)
29+
DISTRIBUTED BY HASH(k1) BUCKETS 1
30+
PROPERTIES (
31+
"enable_unique_key_merge_on_write" = "false",
32+
"disable_auto_compaction" = "true",
33+
"replication_num" = "1"
34+
)
35+
"""
36+
// Three loads make three overlapping rowsets; key (1, 1) has three versions.
37+
sql "INSERT INTO test_read_mor_as_dup_key_topn VALUES (1, 1, 10), (2, 2, 20), (5, 5, 50)"
38+
sql "INSERT INTO test_read_mor_as_dup_key_topn VALUES (1, 1, 11), (3, 3, 30), (6, 6, 60)"
39+
sql "INSERT INTO test_read_mor_as_dup_key_topn VALUES (1, 1, 12), (4, 4, 40), (7, 7, 70)"
40+
41+
// Merged read: one row per key.
42+
qt_merged "SELECT * FROM test_read_mor_as_dup_key_topn ORDER BY k1, k2 LIMIT 3"
43+
44+
sql "SET read_mor_as_dup_tables = 'test_read_mor_as_dup_key_topn'"
45+
// The crash needs segment LIMIT pushdown, which fuzzy sessions may turn off.
46+
sql "SET enable_segment_limit_pushdown = true"
47+
order_qt_dup_asc "SELECT * FROM test_read_mor_as_dup_key_topn ORDER BY k1, k2 LIMIT 4"
48+
qt_dup_desc "SELECT * FROM test_read_mor_as_dup_key_topn ORDER BY k1 DESC, k2 DESC LIMIT 3"
49+
order_qt_dup_prefix "SELECT * FROM test_read_mor_as_dup_key_topn ORDER BY k1 LIMIT 4"
50+
order_qt_dup_filter "SELECT * FROM test_read_mor_as_dup_key_topn WHERE v > 10 ORDER BY k1, k2 LIMIT 3"
51+
qt_dup_offset "SELECT * FROM test_read_mor_as_dup_key_topn ORDER BY k1, k2 LIMIT 2 OFFSET 3"
52+
// A plain LIMIT without ORDER BY still works on the table read as DUP.
53+
order_qt_dup_limit "SELECT count(*) FROM (SELECT * FROM test_read_mor_as_dup_key_topn LIMIT 5) t"
54+
}

0 commit comments

Comments
 (0)