Repository navigation
[Bug](High) ROW binlog rejects valid HLL writes with a misleading QuantileState nullable error #65865
Description
Tracking
Search before asking
- I searched the existing issues and found no issue with the same error or reproduction.
Version
Reproduced while testing #62606 on Doris commit a78b707ed040c3206df99cf43ea3101ab2ba5b90.
Test environment: non-Cloud deployment with 1 FE and 3 BEs.
What's Wrong?
A UNIQUE KEY merge-on-write table can be created with a NOT NULL HLL column and ROW binlog enabled, but the first valid HLL write fails in the BE conversion path:
QuantileState column does not support nullable
The HLL column is declared NOT NULL, and the error incorrectly refers to QuantileState. The same ROW-binlog configuration can write BITMAP values successfully, so the failure is specific to the HLL ROW-binlog write path.
IVM requires ROW binlog, so this blocks IVM scenarios whose base tables contain HLL columns before refresh can begin.
What You Expected?
If HLL is supported with ROW binlog, a valid HLL_HASH value should be written successfully and produce consumable row binlog. If this combination is intentionally unsupported, Doris should reject it during CREATE/ALTER analysis with an accurate error instead of accepting the table and failing every write.
How to Reproduce?
DROP DATABASE IF EXISTS ivm_hll_row_binlog_repro;
CREATE DATABASE ivm_hll_row_binlog_repro;
USE ivm_hll_row_binlog_repro;
CREATE TABLE hll_row_binlog_base (
id BIGINT NOT NULL,
value HLL NOT NULL
)
UNIQUE KEY(id)
DISTRIBUTED BY HASH(id) BUCKETS 1
PROPERTIES (
'replication_num'='1',
'enable_unique_key_merge_on_write'='true',
'binlog.enable'='true',
'binlog.format'='ROW',
'binlog.need_historical_value'='true'
);
INSERT INTO hll_row_binlog_base VALUES (1, HLL_HASH('one'));
SELECT COUNT(*) FROM hll_row_binlog_base;The CREATE succeeds, the INSERT fails with the error above, and the table remains empty.
Regression Test Result
- Suite:
ivm_sql_complex_type_matrix - Jenkins:
regression-debug #792 - Result: CREATE succeeds and the valid
HLL_HASHINSERT fails consistently.
The suite currently records this behavior as a negative capability boundary and therefore passes; that pass means the failure is reproducible, not that the product behavior is correct.
Anything Else?
The failure occurs in the BE HLL column conversion path when a nullmap is present. The returned error text appears to have been copied from the QuantileState converter.
Are you willing to submit PR?
- Yes, I am willing to submit a PR!
Activity
zclllyybb commentedon Jul 22, 2026
Breakwater-GitHub-Analysis-Slot: slot_d3817c3c4dc3
This content is generated by AI for reference only.
Initial triage from the reported commit
a78b707ed040c3206df99cf43ea3101ab2ba5b90and refreshedupstream/master:This looks like a real BE ROW-binlog conversion bug, not invalid SQL. The table creation path accepts this schema: when FE builds the hidden ROW-binlog index,
OlapTable.generateTableRowBinlogSchema()only rejects VARIANT and AUTO_INCREMENT. For non-key value columns it usesColumn.generateAfterValueColumn(), and for historical values it also usesColumn.generateBeforeValueColumn(); both helpers copy the original column and then set the ROW-binlog column nullable. Therefore aNOT NULL HLLbase value can still be written through a nullable ROW-binlog column shape.The failing insert matches the BE path:
RowBinlogSegmentWriter::append_block()callsRowBinlogSourceDataWriter::prepare_by_source_block(), which invokesOlapBlockDataConvertor::convert_column_data()for the source column. ForOLAP_FIELD_TYPE_HLL, the converter isOlapColumnDataConvertorHLL. In that converter, any present_nullmapimmediately returnsStatus::NotSupported("QuantileState column does not support nullable"). So the observed message is a copy/paste error in the HLL branch, and the failure is triggered by the column nullable-wrapper shape, not by QuantileState.Bitmap's converter already handles the nullable-wrapper case by unwrapping
ColumnNullable, serializing non-null rows, and returningnullptr/0slices for null rows. HLL has compatible serialized-slice behavior in the non-null branch, so the likely product fix is to makeOlapColumnDataConvertorHLLhandle nullable HLL like bitmap, including the all-non-null wrapper case used by this repro. If Doris intentionally does not want to support HLL in ROW binlog, the support boundary should instead be enforced in FE create/alter validation before generating the ROW-binlog index, with an HLL-specific message.Suggested next steps:
NOT NULL HLLinsert usingHLL_HASH, and check row-binlog consumption if possible.No additional user data is needed to confirm the primary code path. Runtime logs would only be useful if maintainers want to confirm whether the failing block contains an all-zero nullmap or an actual nullable HLL value.
[fix](binlog) Support nullable HLL conversion for row binlog
[fix](ddl) Restrict state types to aggregate tables
[improvement](ddl) Restrict state types to aggregate tables
[improvement](ddl) Preserve internal state type tables
[improvement](ddl) Preserve state types in async materialized views
[test](regression) Stabilize state type compatibility suites
[chore](ddl) Merge master and resolve state-type validation conflicts
[fix](ddl) Scope state-type exemptions to internal tables
[refactor](ddl) Rename the state-type compatibility setting
[improvement](ddl) Restrict state types to aggregate tables (#66963)