Repository navigation
[BUG] Row binlog tablet schema missing __DORIS_BINLOG_LSN__ column index causing coredump when creating rowset writer #64483
Description
Search before asking
- I had searched in the issues and found no similar issues.
Version
newest master branch
What's Wrong?
Problem Description
When creating a row binlog table and writing data, a coredump occurs in RowBinlogSegmentWriter::init() with the following CHECK failure:
[----------] 2 tests from GroupRowsetWriterTest
19:13:27 [ RUN ] GroupRowsetWriterTest.sub_writer_rollback
19:13:27 F20260613 19:13:27.347458 362558 row_binlog_segment_writer.cpp:69] Check failed: lsn_col_id >= 0 binlog schema missing DORIS_BINLOG_LSN
19:13:27 *** Check failure stack trace: ***
19:13:27 @ 0x55812f5cd876 google::LogMessageFatal::~LogMessageFatal()
19:13:27 @ 0x55812bcd1f9e doris::segment_v2::RowBinlogSegmentWriter::init()
19:13:27 @ 0x55812b79d73e doris::SegmentFlusher::_create_segment_writer()
19:13:27 @ 0x55812b798030 doris::SegmentFlusher::flush_single_block()
19:13:27 @ 0x55812b7a18f1 doris::SegmentCreator::flush_single_block()
19:13:27 @ 0x55812b6dfaae doris::SegmentCreator::flush_single_block()
19:13:27 @ 0x55812b69e591 doris::BaseBetaRowsetWriter::flush_single_block()
19:13:28 @ 0x55812b6f4e2f doris::GroupRowsetWriter::flush_single_block()
19:13:28 @ 0x558114c3a9f7 doris::GroupRowsetWriterTest_sub_writer_rollback_Test::TestBody()
19:13:28 @ 0x55812fcc2f94 testing::internal::HandleSehExceptionsInMethodIfSupported<>()
19:13:28 @ 0x55812fcb1db6 testing::internal::HandleExceptionsInMethodIfSupported<>()
19:13:28 @ 0x55812fc941c7 testing::Test::Run()
19:13:28 @ 0x55812fc94cec testing::TestInfo::Run()
19:13:28 @ 0x55812fc954b2 testing::TestSuite::Run()
19:13:28 @ 0x55812fca6ccc testing::internal::UnitTestImpl::RunAllTests()
19:13:28 @ 0x55812fcc4dc4 testing::internal::HandleSehExceptionsInMethodIfSupported<>()
19:13:28 @ 0x55812fcb3e46 testing::internal::HandleExceptionsInMethodIfSupported<>()
19:13:28 @ 0x55812fca684b testing::UnitTest::Run()
19:13:29 @ 0x558117a84401 RUN_ALL_TESTS()
19:13:29 @ 0x558117a79b0e main
19:13:29 @ 0x7f419b4b3555 __libc_start_main
19:13:29 @ 0x55810c4be02a _start
19:13:29 @ (nil) (unknown)
19:13:29 run-be-ut.sh: line 595: 362558 Aborted (core dumped) LLVM_PROFILE_FILE="${profraw}" "${test}" --gtest_output="xml:${GTEST_OUTPUT_DIR}/${file_name}.xml" --gtest_print_time=true "${FILTER}" "${GEN_OUT}" "${GEN_REGRESSION_CASE}"
Steps to Reproduce
- Run BE unit test:
./run-be-ut.sh --run GroupRowsetWriterTest.sub_writer_rollback
GDB Debug Info
(gdb) p this->_tablet_schema->_binlog_lsn_col_idx
$24 = -1
(gdb) p this->_tablet_schema->_binlog_timestamp_col_idx
$25 = -1
(gdb) p this->_tablet_schema->_num_columns
$26 = 5
The schema contains 5 columns (including 3 binlog special columns), but _binlog_lsn_col_idx is -1.
Steps to Reproduce
cd be/ut_build_dir
./run-be-ut.sh --run GroupRowsetWriterTest.sub_writer_rollback
### Root Cause
When converting TTabletSchema (Thrift) to TabletSchemaPB (Protobuf) via TabletMeta::init_schema_from_thrift(), the indices of binlog special columns (__DORIS_BINLOG_LSN__ and __DORIS_BINLOG_TIMESTAMP__) are not set in the protobuf.
As a result, when TabletSchema::init_from_pb() is called, _binlog_lsn_col_idx remains -1, causing the CHECK failure in RowBinlogSegmentWriter::init().
### Expected Behavior
_binlog_lsn_col_idx should be correctly set to the column index of __DORIS_BINLOG_LSN__ (which is 2 in the test case).
### What You Expected?
successfully executed run-be-ut.sh
### How to Reproduce?
execute run-be-ut.sh will certainly reproduce this problem
### Anything Else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Activity
zclllyybb commentedon Jun 13, 2026
Breakwater-GitHub-Analysis-Slot: slot_c4e837598f90
Initial analysis: this looks like a real row-binlog metadata construction bug on current upstream/master. I checked the refreshed
upstream/masterref at4dda859061c, and the proposed root cause is consistent with the code path.Evidence:
testutil::enable_row_binlog()builds the row-binlog schema by appending__DORIS_BINLOG_LSN__,__DORIS_BINLOG_OP__, and__DORIS_BINLOG_TIMESTAMP__after the normal columns. In the reported test case, the LSN column should therefore be column id 2, and the timestamp column should be column id 4.TabletMeta::init_schema_from_thrift()convertsTTabletSchemadirectly intoTabletSchemaPBby addingColumnPBs. It propagates special indexes that exist onTTabletSchema, such asdelete_sign_idx, but there is no thrift field for the row-binlog LSN/timestamp indexes, and this function does not derive those indexes from the column names.TabletMeta::init_from_pb()then callsTabletSchema::init_from_pb()forrow_binlog_schema. That path assigns_binlog_lsn_col_idxand_binlog_timestamp_col_idxfrom the protobuf fields, whose defaults are-1.RowBinlogSegmentWriter::init()hard-checks_tablet_schema->binlog_lsn_col_idx() >= 0, so it aborts even though the__DORIS_BINLOG_LSN__column exists in the schema's column list.Recommended fix:
TabletMeta::init_schema_from_thrift(), while iteratingtablet_schema.columns, settablet_schema_pb->set_binlog_lsn_col_idx(...)andtablet_schema_pb->set_binlog_timestamp_col_idx(...)whentcolumn.column_namematchesBINLOG_LSN_COLandBINLOG_TIMESTAMP_COL. Alternatively, route this conversion through the normalTabletSchemaappend/to-pb path so all special-column indexes are derived consistently.binlog_lsn_col_idx() == 2andbinlog_timestamp_col_idx() == 4, not only thatfield_index("__DORIS_BINLOG_LSN__") >= 0.field_indexcan pass while the persisted special-column index remains-1.GroupRowsetWriterTest.sub_writer_rollbackas the end-to-end guard; after the metadata fix it should no longer abort inRowBinlogSegmentWriter::init().Missing information:
TabletSchema::init_from_pb()needs a compatibility fallback that derives missing row-binlog indexes from column names when the protobuf fields are still-1.