Skip to content

Commit 369f887

Browse files
[fix](table stream) Fix multi-segment row-binlog reads for MoW tables (#66338)
### What problem does this PR solve? Issue Number: close #65901 Related PR: #65418 Problem Summary: Table Stream queries over ROW binlog could crash or return incorrect results when a row-binlog rowset contained multiple segments. This affected both multi-segment rowsets generated by load and non-singleton overlapping rowsets produced by row-binlog LMax quick merge. Although the source table may use UNIQUE_KEYS with Merge-on-Write, the scan reads from its dedicated DUP_KEYS row-binlog tablet; the problem was caused by inconsistent iterator selection and incorrect ordering and grouping assumptions. For a non-overlapping multi-segment rowset, `force_key_ordered_read` caused `BetaRowsetReader` to build a merge iterator solely because ordered output was required. The outer `VCollectIterator` could still select its block-based union path because the rowset was non-overlapping. This mismatch made the child iterator enter by-reference mode while its parent called the block interface, triggering the `!_get_data_by_ref` assertion and aborting the BE. Non-overlapping segments do not require a merge iterator because an ordered union already preserves key order. Row-binlog LMax quick merge introduces another multi-segment layout: it links segments from multiple rowsets into a non-singleton rowset marked `OVERLAPPING`. The previous `is_segments_overlapping()` implementation only recognized singleton delta rowsets, so these quick-merge rowsets could incorrectly use the union path instead of merging their segments by key and TSO. The TSO column could also be skipped after zone-map pruning even though it is required as the merge sequence column. In addition, MIN_DELTA processing used `IteratorRowRef::is_same` to identify consecutive events belonging to the same user key. This flag is a merge/dedup marker and does not reliably mark same-key events that reside in one segment after quick merge. As a result, one key's change chain could be split into multiple groups and folded incorrectly. This change separates ordered-output requirements from iterator selection: non-overlapping segments use the ordered union path, while segments explicitly marked as overlapping use the merge path. It recognizes non-singleton overlapping rowsets created by quick merge, always materializes the row-binlog TSO column when it is needed for ordering, and groups MIN_DELTA events by comparing their user-key columns directly. Before this change, affected queries could abort the BE, read quick-merge rowsets out of order, or produce incorrect MIN_DELTA results. After this change, multi-segment row-binlog rowsets are read through a consistent iterator interface and processed in key/TSO order with correct per-key change folding.
1 parent 6de4f85 commit 369f887

17 files changed

Lines changed: 1800 additions & 17 deletions

‎be/src/storage/iterator/block_reader.cpp‎

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -216,6 +216,11 @@ Status BlockReader::_min_delta_next_block(Block* block, bool* eof) {
216216
const int32_t tso_ordinal = _read_schema->tso_ordinal();
217217
const int32_t lsn_ordinal = _read_schema->lsn_ordinal();
218218
const int32_t op_ordinal = _read_schema->op_ordinal();
219+
// A group is a run of consecutive rows sharing the same user key. Row-binlog reads are
220+
// globally key-ordered (ReaderParams::force_key_ordered_read), and the key columns are the
221+
// leading num_key_columns() columns of every read block, so a group boundary is exactly
222+
// where the user key changes. _stored_data_columns keeps the group's first row at index 0.
223+
const size_t num_key_columns = _tablet_schema->num_key_columns();
219224
while (output_row_count < batch_max_rows()) {
220225
if (_emit_pending_row(target_columns, output_row_count)) {
221226
continue;
@@ -238,8 +243,27 @@ Status BlockReader::_min_delta_next_block(Block* block, bool* eof) {
238243
return res;
239244
}
240245

241-
if (!_eof && _next_row.is_same) {
242-
continue;
246+
// Extend the current group while the next row shares the same user key. is_same cannot
247+
// be used here: it marks cross-segment key matches for dedup, so consecutive same-key
248+
// rows that a compaction/quick-merge folded into one segment are left unmarked, which
249+
// would split one key's change chain into several groups. Compare the leading key
250+
// columns directly against the group's first row (index 0 of _stored_data_columns).
251+
if (!_eof) {
252+
if (_next_row.is_same) {
253+
continue;
254+
}
255+
bool same_key = true;
256+
for (size_t k = 0; k < num_key_columns; ++k) {
257+
if (_stored_data_columns[k]->compare_at(0, _next_row.row_pos,
258+
*_next_row.block->get_by_position(k).column,
259+
-1) != 0) {
260+
same_key = false;
261+
break;
262+
}
263+
}
264+
if (same_key) {
265+
continue;
266+
}
243267
}
244268
size_t group_size = _stored_data_columns[0]->size();
245269
auto first_op = _read_binlog_op(*_stored_data_columns[op_ordinal], 0);

‎be/src/storage/merger.cpp‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,12 @@ Status Merger::vmerge_rowsets(BaseTabletSPtr tablet, ReaderType reader_type,
7777
reader_params.tablet = tablet;
7878
reader_params.reader_type = reader_type;
7979
reader_params.read_row_binlog = tablet->is_row_binlog_tablet();
80+
if (reader_params.read_row_binlog) {
81+
// Row-binlog horizontal (non-vertical) compaction must produce a globally
82+
// (key, TSO)-ordered output.
83+
reader_params.read_orderby_key = true;
84+
reader_params.force_key_ordered_read = true;
85+
}
8086

8187
TabletReadSource read_source;
8288
read_source.rs_splits.reserve(src_rowset_readers.size());

‎be/src/storage/rowset/beta_rowset_reader.h‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -61,8 +61,7 @@ class BetaRowsetReader : public RowsetReader {
6161

6262
bool is_merge_iterator() const override {
6363
return _read_context->need_ordered_result && _get_segment_num() > 1 &&
64-
(_rowset->rowset_meta()->is_segments_overlapping() ||
65-
_read_context->force_key_ordered_read);
64+
_rowset->rowset_meta()->is_segments_overlapping();
6665
}
6766

6867
bool delete_flag() override { return _rowset->delete_flag(); }

‎be/src/storage/rowset/beta_rowset_writer.cpp‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1074,6 +1074,17 @@ Status BaseBetaRowsetWriter::_build_rowset_meta(RowsetMeta* rowset_meta, bool ch
10741074
!is_segment_overlapping(segments_encoded_key_bounds) &&
10751075
_context.tablet_schema->cluster_key_uids().empty()) {
10761076
rowset_meta->set_segments_overlap(NONOVERLAPPING);
1077+
} else if (!segments_encoded_key_bounds.empty() &&
1078+
is_segment_overlapping(segments_encoded_key_bounds) &&
1079+
_rowset_meta->is_row_binlog()) {
1080+
// A row-binlog horizontal merge output can hold the same user key in more than
1081+
// one output segment (a forced segment boundary splits a key range), so the
1082+
// segments genuinely overlap. The compaction ctx default is NONOVERLAPPING for
1083+
// the non-quick-merge path, and the branch above only ever downgrades, so upgrade
1084+
// here. Otherwise RowsetMeta::is_segments_overlapping() and
1085+
// BetaRowsetReader::is_merge_iterator() would pick a segment union instead of a
1086+
// merge and break downstream MIN_DELTA reads.
1087+
rowset_meta->set_segments_overlap(OVERLAPPING);
10771088
}
10781089

10791090
auto segment_num = _num_seg();

‎be/src/storage/rowset/rowset_meta.h‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -306,8 +306,11 @@ class RowsetMeta : public MetadataAdder<RowsetMeta> {
306306
// 2. the rowset's start version == end version (non-singleton rowset was generated by compaction process
307307
// which always produces non-overlapped segments)
308308
// 3. segments_overlap() flag is not NONOVERLAPPING (OVERLAP_UNKNOWN and OVERLAPPING are OK)
309+
// 4. relaxing is_singleton_delta() for is_row_binlog as Row-binlog LMax quick merge does produce
310+
// non-singleton overlapping segments, but it writes segments_overlap = OVERLAPPING explicitly
309311
bool is_segments_overlapping() const {
310-
return num_segments() > 1 && is_singleton_delta() && segments_overlap() != NONOVERLAPPING;
312+
return num_segments() > 1 && segments_overlap() != NONOVERLAPPING &&
313+
(is_singleton_delta() || (is_row_binlog() && segments_overlap() == OVERLAPPING));
311314
}
312315

313316
bool produced_by_compaction() const {

‎be/src/storage/rowset/rowset_reader_context.h‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,6 @@ struct RowsetReaderContext {
5252
// For rows with the same key, use ascending order (small-to-large) for tie-breakers.
5353
// For example, use lower rowset version / segment id first.
5454
bool use_insert_order_when_same = false;
55-
bool force_key_ordered_read = false;
5655
// ordinals (positions in read_schema) of the orderby key columns
5756
std::vector<uint32_t>* read_orderby_key_columns = nullptr;
5857
// limit of rows for read_orderby_key

‎be/src/storage/segment/segment_iterator.cpp‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1561,6 +1561,15 @@ bool SegmentIterator::_need_read_data(ColumnId cid) {
15611561
// occurring, return true here that column data needs to be read
15621562
return true;
15631563
}
1564+
// Row-binlog incremental reads force-push the TSO range predicate (see
1565+
// OlapScanner::_init_tso_predicate), and the merge iterator uses the TSO column as its
1566+
// sequence sort key (BetaRowsetReader sets binlog_tso_idx). On a cross-version rowset
1567+
// (e.g. produced by binlog LMax quick-merge) the TSO zonemap can be always-true, so the
1568+
// pruning below would skip reading it and fill placeholder zeros, breaking the merge
1569+
// ordering. The TSO column carries real values on disk, so force it to be read.
1570+
if (_opts.read_row_binlog && cid == _schema->tso_ordinal()) {
1571+
return true;
1572+
}
15641573
const auto& column = *_schema->column(cid);
15651574
// Different subcolumns may share the same parent_unique_id, so we choose to abandon this optimization.
15661575
if (column.is_extracted_column() &&

‎be/src/storage/tablet/tablet_reader.cpp‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -159,12 +159,11 @@ Status TabletReader::_capture_rs_readers(const ReaderParams& read_params) {
159159
_reader_context.read_row_binlog = read_params.read_row_binlog;
160160
_reader_context.version = read_params.version;
161161
_reader_context.tablet_schema = _tablet_schema;
162-
_reader_context.need_ordered_result = need_ordered_result;
162+
_reader_context.need_ordered_result = need_ordered_result || read_params.force_key_ordered_read;
163163
_reader_context.topn_filter_source_node_ids = read_params.topn_filter_source_node_ids;
164164
_reader_context.read_orderby_key_reverse = read_params.read_orderby_key_reverse;
165165
_reader_context.use_insert_order_when_same =
166166
read_params.use_insert_order_when_same || read_params.read_row_binlog;
167-
_reader_context.force_key_ordered_read = read_params.force_key_ordered_read;
168167
_reader_context.read_orderby_key_limit = read_params.read_orderby_key_limit;
169168
_reader_context.read_schema = _read_schema;
170169
_reader_context.read_orderby_key_columns =

‎be/src/storage/tablet/tablet_reader.h‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -183,10 +183,9 @@ class TabletReader {
183183
// For rows with the same key, use ascending order (small-to-large) for tie-breakers.
184184
// For example, use lower rowset version / segment id first.
185185
bool use_insert_order_when_same = false;
186-
// Force a key-ordered merge across all segments even when their key ranges do not
187-
// overlap. By default a rowset reader can skip the merge heap if its segments are
188-
// mono-ascending and disjoint, but row-binlog scans require strict global key order
189-
// (e.g. so MIN_DELTA can group consecutive same-key changes), so this flag is set.
186+
// Force globally key-ordered reading for row-binlog scans (e.g. so MIN_DELTA can
187+
// group consecutive same-key changes across segments). Overlapping segments use the
188+
// merge iterator; segments proven non-overlapping use an ordered union.
190189
// See BetaRowsetReader::is_merge_iterator() in beta_rowset_reader.h:62.
191190
bool force_key_ordered_read = false;
192191
// num of columns for orderby key

‎be/test/exec/scan/vgeneric_iterators_test.cpp‎

Lines changed: 163 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ class VGenericIteratorsTest : public testing::Test {
5050
virtual ~VGenericIteratorsTest() {}
5151
};
5252

53-
static ReadSchema create_schema() {
53+
static std::vector<TabletColumnPtr> create_col_schemas() {
5454
std::vector<TabletColumnPtr> col_schemas;
5555
auto c1 = std::make_shared<TabletColumn>(FieldAggregationMethod::OLAP_FIELD_AGGREGATION_NONE,
5656
FieldType::OLAP_FIELD_TYPE_SMALLINT, true);
@@ -65,7 +65,11 @@ static ReadSchema create_schema() {
6565
col_schemas.emplace_back(
6666
std::make_shared<TabletColumn>(FieldAggregationMethod::OLAP_FIELD_AGGREGATION_SUM,
6767
FieldType::OLAP_FIELD_TYPE_BIGINT, true));
68-
return ReadSchema(std::move(col_schemas));
68+
return col_schemas;
69+
}
70+
71+
static ReadSchema create_schema() {
72+
return ReadSchema(create_col_schemas());
6973
}
7074

7175
static void create_block(ReadSchema& schema, Block& block) {
@@ -474,4 +478,161 @@ TEST(VGenericIteratorsTest, MergeWithSeqColumnSmallSeqFirst) {
474478
EXPECT_EQ(0, actual_value);
475479
}
476480

481+
// Models the row-binlog quick-merge read path: the merge output rowset holds the
482+
// same user key across multiple segments, each carrying a distinct binlog event
483+
// (distinct TSO). A row-binlog scan reads it with is_unique=false and the TSO as
484+
// the sequence column with small_seq_first=true. Unlike the is_unique=true cases
485+
// above (which collapse same-key rows to one), the merge iterator must keep EVERY
486+
// event and emit them ordered by ascending TSO, so a table-stream query over a
487+
// quick-merge rowset returns the complete, ordered change chain without dropping
488+
// any event.
489+
TEST(VGenericIteratorsTest, MergeKeepsAllBinlogEventsOrderedByTso) {
490+
auto schema = create_schema();
491+
auto output_schema = std::make_shared<ReadSchema>(schema);
492+
std::vector<RowwiseIteratorUPtr> inputs;
493+
494+
int seq_column_id = 2; // BIGINT column stands in for __DORIS_BINLOG_TSO__.
495+
int num_rows = 1;
496+
int rows_begin = 0;
497+
// Same key in every segment, each carrying a distinct TSO. The children are fed in a
498+
// deterministic SHUFFLED TSO order (not ascending), so the test only passes if the
499+
// merge actually orders by the TSO sequence column; a comparator that ignored the
500+
// sequence and fell back to child/data-id arrival order would emit this shuffled
501+
// sequence unchanged and fail the ascending-order assertion below.
502+
const std::vector<int> tso_feed_order = {7, 1, 9, 0, 5, 8, 2, 6, 4, 3};
503+
int seg_iter_num = static_cast<int>(tso_feed_order.size());
504+
for (int tso : tso_feed_order) {
505+
inputs.push_back(std::make_unique<SeqColumnUtIterator>(schema, num_rows, rows_begin,
506+
seq_column_id, tso));
507+
}
508+
509+
// is_unique=false keeps every same-key event; small_seq_first=true orders by
510+
// ascending TSO. This mirrors beta_rowset_reader's row-binlog merge setup.
511+
auto iter = new_merge_iterator(std::move(inputs), seq_column_id, /*is_unique=*/false,
512+
/*is_reverse=*/false, /*merged_rows=*/nullptr, output_schema,
513+
/*small_seq_first=*/true);
514+
StorageReadOptions opts;
515+
auto st = iter->init(opts);
516+
EXPECT_TRUE(st.ok());
517+
518+
Block block;
519+
std::vector<bool> row_is_same;
520+
BlockWithSameBit block_with_same_bit {.block = &block, .same_bit = row_is_same};
521+
create_block(schema, block);
522+
523+
do {
524+
st = iter->next_batch(&block_with_same_bit);
525+
} while (st.ok());
526+
527+
EXPECT_TRUE(st.is<END_OF_FILE>());
528+
// Every event survives: no per-key dedup.
529+
EXPECT_EQ(block.rows(), seg_iter_num);
530+
531+
// Despite the shuffled input order, events are emitted in ascending TSO order.
532+
auto seq_col = block.get_by_position(seq_column_id).column;
533+
for (int i = 0; i < seg_iter_num; ++i) {
534+
EXPECT_EQ(i, (*seq_col)[i].get<TYPE_BIGINT>());
535+
}
536+
}
537+
538+
// Emits num_rows rows for a schema whose projection (col_ids) may be narrower than the
539+
// full tablet schema. The filled block layout follows the projection, matching how
540+
// VMergeIteratorContext::block_reset builds its block from the output schema.
541+
class ProjectedColumnsUtIterator : public RowwiseIterator {
542+
public:
543+
ProjectedColumnsUtIterator(ReadSchema schema, size_t num_rows)
544+
: _schema(std::move(schema)), _num_rows(num_rows) {}
545+
~ProjectedColumnsUtIterator() override = default;
546+
547+
Status init(const StorageReadOptions& opts) override { return Status::OK(); }
548+
549+
Status next_batch(Block* block) override {
550+
if (_rows_returned >= _num_rows) {
551+
return Status::EndOfFile("End of ProjectedColumnsUtIterator");
552+
}
553+
while (_rows_returned < _num_rows) {
554+
for (size_t j = 0; j < _schema.num_block_columns(); ++j) {
555+
ColumnWithTypeAndName& vc = block->get_by_position(j);
556+
IColumn& vi = (IColumn&)(*vc.column);
557+
558+
char data[16] = {};
559+
size_t data_len = 0;
560+
const auto* col_schema = _schema.column(j);
561+
switch (col_schema->type()) {
562+
case FieldType::OLAP_FIELD_TYPE_SMALLINT:
563+
*(int16_t*)data = static_cast<int16_t>(_rows_returned);
564+
data_len = sizeof(int16_t);
565+
break;
566+
case FieldType::OLAP_FIELD_TYPE_INT:
567+
*(int32_t*)data = static_cast<int32_t>(_rows_returned);
568+
data_len = sizeof(int32_t);
569+
break;
570+
case FieldType::OLAP_FIELD_TYPE_BIGINT:
571+
*(int64_t*)data = static_cast<int64_t>(_rows_returned);
572+
data_len = sizeof(int64_t);
573+
break;
574+
default:
575+
break;
576+
}
577+
578+
vi.insert_data(data, data_len);
579+
}
580+
++_rows_returned;
581+
}
582+
return Status::OK();
583+
}
584+
585+
const ReadSchema& schema() const override { return _schema; }
586+
587+
private:
588+
ReadSchema _schema;
589+
size_t _num_rows;
590+
size_t _rows_returned = 0;
591+
};
592+
593+
// The merge-heap comparator compares the leading num_key_columns() block positions when no
594+
// explicit compare columns are given. The read schema must therefore place its key columns as
595+
// the leading prefix; if a non-key column precedes a key column, init() must reject the merge
596+
// with a "compare contract violated" error instead of silently comparing a value column as a key
597+
// (issue #66390: a ROW-binlog APPEND_ONLY scan projected value columns ahead of a key).
598+
TEST(VGenericIteratorsTest, MergeRejectsProjectionMissingKeyPrefix) {
599+
// Full tablet schema: k0(smallint), k1(int), v2(bigint). Project {v2, k0}: the schema has
600+
// one key column (k0) but it is not the leading column, so ordinal 0 is a non-key column.
601+
ReadSchema projected(
602+
project_columns_by_ordinal(create_col_schemas(), std::vector<ColumnId> {2, 0}));
603+
auto output_schema = std::make_shared<ReadSchema>(projected);
604+
605+
std::vector<RowwiseIteratorUPtr> inputs;
606+
inputs.push_back(std::make_unique<ProjectedColumnsUtIterator>(projected, 10));
607+
inputs.push_back(std::make_unique<ProjectedColumnsUtIterator>(projected, 10));
608+
609+
auto iter = new_merge_iterator(std::move(inputs), -1, false, false, nullptr, output_schema);
610+
StorageReadOptions opts;
611+
auto st = iter->init(opts);
612+
EXPECT_FALSE(st.ok());
613+
EXPECT_TRUE(st.to_string().find("compare contract violated") != std::string::npos)
614+
<< st.to_string();
615+
}
616+
617+
// Same contract, a different non-leading-key layout: project {v2, k1}. num_key_columns() is 1
618+
// (k1), but position 0 is the value column v2, so the merge comparator would compare v2 as if it
619+
// were the leading key. init() must reject it.
620+
TEST(VGenericIteratorsTest, MergeRejectsProjectionWithoutLeadingKey) {
621+
// Full tablet schema: k0(smallint), k1(int), v2(bigint); project {v2, k1}.
622+
ReadSchema projected(
623+
project_columns_by_ordinal(create_col_schemas(), std::vector<ColumnId> {2, 1}));
624+
auto output_schema = std::make_shared<ReadSchema>(projected);
625+
626+
std::vector<RowwiseIteratorUPtr> inputs;
627+
inputs.push_back(std::make_unique<ProjectedColumnsUtIterator>(projected, 10));
628+
inputs.push_back(std::make_unique<ProjectedColumnsUtIterator>(projected, 10));
629+
630+
auto iter = new_merge_iterator(std::move(inputs), -1, false, false, nullptr, output_schema);
631+
StorageReadOptions opts;
632+
auto st = iter->init(opts);
633+
EXPECT_FALSE(st.ok());
634+
EXPECT_TRUE(st.to_string().find("compare contract violated") != std::string::npos)
635+
<< st.to_string();
636+
}
637+
477638
} // namespace doris

0 commit comments

Comments
 (0)