Skip to content

Commit 4dc0cdd

Browse files
committed
docs: note that INTERSECT ALL and EXCEPT ALL plans cannot be exported to Substrait
The Substrait producer cannot serialize Unnest, which these plans now contain. Document the failure in the 56.0.0 upgrade guide and link apache#26372 for supporting it. Signed-off-by: 1fanwang <[email protected]>
1 parent 7bed929 commit 4dc0cdd

1 file changed

Lines changed: 11 additions & 0 deletions

File tree

  • docs/source/library-user-guide/upgrading

‎docs/source/library-user-guide/upgrading/56.0.0.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -910,6 +910,17 @@ unchanged and still return the type of the left input.
910910
counts. Use `LogicalPlanBuilder::intersect_all` and
911911
`LogicalPlanBuilder::except_all` instead.
912912

913+
### `INTERSECT ALL` and `EXCEPT ALL` plans cannot be exported to Substrait
914+
915+
The Substrait producer cannot serialize `LogicalPlan::Unnest`, so converting a
916+
plan from SQL `INTERSECT ALL` or `EXCEPT ALL`, `DataFrame::intersect`, or
917+
`DataFrame::except` to Substrait now fails with `Unsupported plan type: Unnest`.
918+
Earlier releases exported a semi join or an anti join for these plans, which
919+
returned the wrong number of duplicate rows, so they now fail instead of
920+
exporting a plan with different results. Substrait plans that contain multiset
921+
set operations are still consumed. Exporting `Unnest` is tracked in
922+
<https://github.com/apache/datafusion/issues/26372>.
923+
913924
### Null-aware joins record their `NOT IN` value key count
914925

915926
Multi-column `(a, b) NOT IN (SELECT x, y ...)` is now planned as a null-aware

0 commit comments

Comments
 (0)