Repository navigation
Commit f0b2128
[improvement](fe) Reject only lossy casts that reach a Variant leaf of the write target
### What problem does this PR solve?
Issue Number: None
Related PR: #69194
Problem Summary: VariantWritePlanValidator (#69194) follows the lineage of
every target column that contains a Variant and rejects an implicit
Variant-to-non-Variant cast on the way, because common-type analysis can turn
an object or array Variant into SQL NULL before the sink encodes it back. It
traced the whole target column and every argument of every expression, so it
also rejected writes that lose nothing:
- a scalar field of a composite target computed from a Variant, such as
named_struct('matched', COALESCE(v['n'], 0) > 1, 'payload', v) into
STRUCT<matched BOOLEAN, payload VARIANT>;
- a Variant that is only compared or only picks an element: v['n'] > 0,
NULLIF(1, IF(TRUE, v, 1)), the index or key of element_at.
The validator now traces a demand per ExprId: which parts of a value (none,
all, or per struct field / array element / map key and value) reach a Variant
leaf. The target type gives the demand, aligned position by position to the
source type, and all of a value of another shape is traced. A closed set of
structural expressions passes the demand on (named_struct, struct, array, map,
element_at, casts between values of the same shape, the results of IF / CASE /
NULLIF / NVL, aliases, window functions, set operations, CTEs and scalar
subqueries); any other expression traces all of its arguments as before. The
second argument of NULLIF, the index or key of element_at, and the implicit
cast a comparison, IN, LIKE / REGEXP, AND / OR or NOT applies directly to an
operand are not traced; the operand under that cast still is, so
IF(TRUE, v, 1) > 0 stays rejected, as do ELT, SUBSTRING and MAX_BY arguments.
A CTE output is traced again only for a part not traced yet. The error now
suggests an explicit CAST.
### Release note
Writes into an external table column that contains a VARIANT no longer fail
when a Variant is implicitly cast only to compute a non-Variant field of the
target (for example the BOOLEAN field of a STRUCT), only to be compared
(comparison, IN, LIKE, REGEXP, AND, OR, NOT), or only to choose a value (the
second argument of NULLIF, the index or key of element_at). The error for a
lossy cast now suggests an explicit CAST.
### Check List (For Author)
- Test: Regression test / Unit Test
- Unit Test: VariantWritePlanValidatorTest, 6 new tests; 19 mutations of
the new rules each fail at least one test
- Regression test: test_paimon_write_variant_errors (new struct and
comparison cases), run locally against the iceberg docker environment
- Behavior changed: Yes (the writes in the release note are accepted; the
error message gains a hint)
- Does this need documentation: No
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>1 parent cefe023 commit f0b2128
4 files changed
Lines changed: 695 additions & 160 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris/datasource
- test/java/org/apache/doris/datasource
- regression-test
- data/external_table_p0/paimon/write
- suites/external_table_p0/paimon/write
0 commit comments