Repository navigation
Commit aaca944
committed
address review: make native cartesian product opt-in (default false)
DataFusion's CrossJoinExec buffers the whole left input and cannot spill, so a
large left can OOM where Spark's spillable CartesianProduct succeeds (sunchao's
P2). Default the flag to false until the native path can spill; add a regression
test that a cross join stays on Spark by default.
Part of apache#199.1 parent 9c9292a commit aaca944
2 files changed
Lines changed: 28 additions & 1 deletion
File tree
- spark/src
- main/scala/org/apache/comet
- test/scala/org/apache/comet/exec
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
362 | 362 | | |
363 | 363 | | |
364 | 364 | | |
365 | | - | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
366 | 373 | | |
367 | 374 | | |
368 | 375 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
188 | 188 | | |
189 | 189 | | |
190 | 190 | | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
191 | 211 | | |
192 | 212 | | |
193 | 213 | | |
| |||
0 commit comments