You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit ed17894
Browse filesBrowse the repository at this point in the historyBrowse files
committed
[test](external) Cover connector INSERT OVERWRITE of a source the planner proves empty
### What problem does this PR solve?
Issue Number: None
Related PR: #69010, #62344
Problem Summary: When the planner folds the source of an INSERT OVERWRITE to an
empty relation (LIMIT 0, WHERE false) and no exchange separates the sink from it,
the insert is an empty insert. Before #69010 an empty insert skipped its
transaction, so such an overwrite returned OK and the target kept all its rows.
branch-4.1 #62344 runs that transaction only for an Iceberg static partition
overwrite; master #69010 runs it for every connector overwrite
(PluginDrivenInsertExecutor.requiresTransaction). Master behaves correctly, but
no test covers it:
- PluginDrivenInsertExecutorTest has no case for requiresTransaction.
- The planner-proven empty overwrite in test_iceberg_partition_overwrite_mode
writes a partitioned table, which master routes through a hash exchange, so
the writers run and the case passes without the #69010 override.
- The LIMIT 0 overwrite in test_paimon_write_transaction writes an append-only
table, which goes through an exchange as well.
The sink sits directly on the empty relation for a Paimon primary-key table
without a fixed bucket count (Paimon's default, written by a single writer), and
for an unpartitioned Iceberg table when enable_strict_consistency_dml is false
(the default in cloud mode). On a local cluster with the #69010 override
removed, a LIMIT 0 overwrite of such a Paimon table, of a static partition of
one, and of such an Iceberg table returned OK and left every row in place. On
master all three clear their target.
This adds:
- PluginDrivenInsertExecutorTest: an empty overwrite, of the whole table or of a
static partition, requires its transaction; an empty INSERT does not.
- PaimonConnectorTransactionTest: an overwrite that receives no commit data
replaces its static partition, or the whole table, with nothing.
- IcebergConnectorTransactionTest: the same for an Iceberg static partition,
ported from the branch-4.1 test of #62344.
- test_paimon_write_transaction: LIMIT 0 overwrites of a dynamic-bucket
primary-key table and of a static partition of one, with the plan that puts
the sink on the empty relation. The comment of the append-only case now says
that it goes through an exchange.
- test_iceberg_insert_overwrite_with_empty_table: a LIMIT 0 overwrite of an
unpartitioned table with strict DML consistency off.
### Release note
None
### Check List (For Author)
- Test: Unit Test, Regression test
- PluginDrivenInsertExecutorTest (run-fe-ut.sh), PaimonConnectorTransactionTest
and IcebergConnectorTransactionTest pass; each new case fails on a mutation
of the code it covers.
- test_paimon_write_transaction and test_iceberg_insert_overwrite_with_empty_table
pass on a local cluster against the iceberg docker environment, and fail
with the #69010 override removed.
- Behavior changed: No
- Does this need documentation: No
Co-authored-by: Socrates <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergConnectorTransactionTest.java
+24Lines changed: 24 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -961,6 +961,30 @@ public void overwriteStaticPartitionUsesRowFilter() {
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-paimon/src/test/java/org/apache/doris/connector/paimon/PaimonConnectorTransactionTest.java
Copy file name to clipboardExpand all lines: fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/insert/PluginDrivenInsertExecutorTest.java
Copy file name to clipboardExpand all lines: regression-test/suites/external_table_p0/iceberg/write/test_iceberg_insert_overwrite_with_empty_table.groovy
0 commit comments