Skip to content

Commit bb32213

Browse files
committed
fix order by key topn slow since read unecessary data (apache#1197)
since key is already sorted and topn_next only need first N rows from each segment, but runtime predicate will filter some rows and read more than N rows. should add add for order by none-key column, since none-key column is not sorted and all rows should be read, so runtime predicate will reduce rows for topn node
1 parent fc2a536 commit bb32213

1 file changed

Lines changed: 9 additions & 3 deletions

File tree

‎be/src/olap/rowset/segment_v2/segment_iterator.cpp‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -969,9 +969,15 @@ void SegmentIterator::_vec_init_lazy_materialization() {
969969
}
970970

971971
// add runtime predicate to _col_predicates
972-
if (_opts.use_topn_opt) {
973-
auto& runtime_predicate =
974-
_opts.runtime_state->get_query_fragments_ctx()->get_runtime_predicate();
972+
// should NOT add for order by key,
973+
// since key is already sorted and topn_next only need first N rows from each segment,
974+
// but runtime predicate will filter some rows and read more than N rows.
975+
// should add add for order by none-key column, since none-key column is not sorted and
976+
// all rows should be read, so runtime predicate will reduce rows for topn node
977+
if (_opts.use_topn_opt &&
978+
!(_opts.read_orderby_key_columns != nullptr && !_opts.read_orderby_key_columns->empty())) {
979+
auto & runtime_predicate =
980+
_opts.runtime_state->get_query_fragments_ctx()->get_runtime_predicate();
975981
_runtime_predicate = runtime_predicate.get_predictate();
976982
if (_runtime_predicate) {
977983
_col_predicates.push_back(_runtime_predicate.get());

0 commit comments

Comments
 (0)