Repository navigation
Commit f9a39fe
committed
address review: make native cartesian product opt-in (default false)
DataFusion's CrossJoinExec buffers the whole left input and cannot spill, so a
large left can OOM where Spark's spillable CartesianProduct succeeds (sunchao's
P2). Default the flag to false until the native path can spill; add a regression
test that a cross join stays on Spark by default.
Part of apache#199.1 parent 5db5de8 commit f9a39fe
2 files changed
Lines changed: 28 additions & 1 deletion
File tree
- spark/src
- main/scala/org/apache/comet
- test/scala/org/apache/comet/exec
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
320 | 320 | | |
321 | 321 | | |
322 | 322 | | |
323 | | - | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
324 | 331 | | |
325 | 332 | | |
326 | 333 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
184 | 184 | | |
185 | 185 | | |
186 | 186 | | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
187 | 207 | | |
188 | 208 | | |
189 | 209 | | |
| |||
0 commit comments