You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 3f31e33
Browse filesBrowse the repository at this point in the historyBrowse files
-**Typed Dataset operators** (`DeserializeToObjectExec`, `SerializeFromObjectExec`, `MapElementsExec`, `MapPartitionsExec`, `MapGroupsExec`, `CoGroupExec`, `AppendColumnsExec`, and similar): produced by `map`, `mapPartitions`, `groupByKey`, `cogroup`, and other typed `Dataset` transformations. They exist to run user JVM functions on JVM objects, which Comet cannot do natively. The operators above them can still run natively: set `spark.comet.convert.typedDataset.enabled=true` and Comet converts the output of a typed operation to Arrow. This is disabled by default because it can be slower than Spark when the operators above do little work, such as an aggregate over a few groups, which Spark compiles together with the typed operation into one loop.
59
58
@@ -97,12 +96,13 @@ omitted from the tables below and may be reconsidered based on demand:
|`BroadcastHashJoinExec`| ✅ | Falls back to Spark for a `NOT IN` (null-aware anti join) whose streamed side reports an ordering. |
103
-
|`ShuffledHashJoinExec`| ✅ ||
104
-
|`SortMergeJoinExec`| ✅ | Supports `BINARY` join keys. Nested-type (struct, array, map) and collated-string join keys fall back to Spark. |
105
-
|`BroadcastNestedLoopJoinExec`| ✅ | Falls back to Spark when the preserved side is broadcast (for example LEFT OUTER with BROADCAST on the left) ([#4429](https://github.com/apache/datafusion-comet/pull/4429)). |
|`BroadcastHashJoinExec`| ✅ | Falls back to Spark for a `NOT IN` (null-aware anti join) whose streamed side reports an ordering. |
102
+
|`ShuffledHashJoinExec`| ✅ ||
103
+
|`SortMergeJoinExec`| ✅ | Supports `BINARY` join keys. Nested-type (struct, array, map) and collated-string join keys fall back to Spark. |
104
+
|`BroadcastNestedLoopJoinExec`| ✅ | Falls back to Spark when the preserved side is broadcast (for example LEFT OUTER with BROADCAST on the left) ([#4429](https://github.com/apache/datafusion-comet/pull/4429)). |
105
+
|`CartesianProductExec`| ⚠️ | Opt-in (`spark.comet.exec.cartesianProduct.enabled`). Runs natively only without a join condition and when each input reports exactly one partition. Row order can differ from Spark's. |
106
106
107
107
`ExistenceJoin` (produced when `EXISTS` / `IN` is combined with another predicate via `OR`) runs
108
108
natively on `BroadcastHashJoinExec` and `ShuffledHashJoinExec`. Existence sort-merge joins, residual
0 commit comments