You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2c95fb1
Browse filesBrowse the repository at this point in the historyBrowse files
[fix](paimon) Cast binary static partition values to columns of any length
### What problem does this PR solve?
Issue Number: None
Related PR: #68874
Problem Summary: Paimon BINARY(n) and VARBINARY(n) columns map to Doris
VARBINARY(n), while a binary literal such as X'4142' is VARBINARY of the
maximum length. PluginDrivenInsertCommandContext.castStaticPartitionSpec
cast a static partition value with Literal.checkedCastTo, which casts a
VARBINARY to no other length, so INSERT OVERWRITE ... PARTITION
(p = X'4142') on such a column failed while it was planned with "Cast from
4142 to VARBINARY(2) not supported". Only BYTES columns, whose type is
VARBINARY of the maximum length, reached the decoding of the previous
commit. The rows of the same statement are written with their bytes:
BindSink leaves their cast to BE, which keeps them, since a VARBINARY
length is metadata only.
The cast value of a binary literal now keeps its bytes and takes the
column's VARBINARY type, as the rows do. Paimon parses the static value
back to those bytes for a BINARY(n) column whatever their length, as the
rows carry them; for a VARBINARY(n) column it fails the commit, after the
BE wrote its files, when the bytes are longer than n, so such a value is
now rejected while the write is planned, naming the column, the value and
its length.
The Javadoc of padChar, which the previous commit had separated from its
method, is moved back.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- PluginDrivenTableSinkBindingTest: X'41', X'4142' and X'414243' keep their bytes when cast for a VARBINARY(2) column through the sink binding
- PaimonWriteBindingTest: a BINARY(2) static value keeps one or three bytes, and a VARBINARY(2) value longer than two bytes is rejected
- IcebergStaticPartitionLiteralTest, PluginDrivenTableSinkTest, fe-connector-paimon module tests
- Behavior changed: Yes. INSERT OVERWRITE ... PARTITION (p = X'...') on a Paimon BINARY(n) / VARBINARY(n) partition column is planned instead of failing on the cast; a value longer than a VARBINARY(n) column fails while planned instead of at commit.
- Does this need documentation: No
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonWriteBinding.java
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-paimon/src/test/java/org/apache/doris/connector/paimon/PaimonWriteBindingTest.java
+28Lines changed: 28 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -44,6 +44,7 @@
44
44
importjava.time.OffsetDateTime;
45
45
importjava.util.Arrays;
46
46
importjava.util.Collections;
47
+
importjava.util.HexFormat;
47
48
importjava.util.LinkedHashMap;
48
49
importjava.util.List;
49
50
importjava.util.Map;
@@ -232,6 +233,33 @@ public void staticBinaryPartitionRejectsBytesThatAreNotUtf8(@TempDir Path wareho
Copy file name to clipboardExpand all lines: fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/PluginDrivenInsertCommandContext.java
0 commit comments