Repository navigation
Commit cc7565b
authored
fix: don't report equi-key equivalence for PiecewiseMergeJoin range joins (apache#24361)
## Which issue does this PR close?
- Closes apache#24360.
## Rationale for this change
`PiecewiseMergeJoinExec::compute_properties` passed the join's `on` pair
to `join_equivalence_properties` as if it were an equijoin key. For an
`INNER` join that registers `left_on == right_on` as an output
equivalence — but PWMJ's `on` is a **range** predicate (`l.v < r.v`),
not equality, so the equivalence is false. It let the optimizer treat a
sort on the left key as also sorting the right key and drop a required
`ORDER BY`, returning wrongly ordered rows.
```sql
set datafusion.optimizer.enable_piecewise_merge_join = true;
create table l(v int) as values (1),(2),(3),(5),(8);
create table r(v int) as values (4),(6),(9),(2);
select l.v, r.v from l join r on l.v < r.v where l.v = 2 order by r.v;
-- PWMJ: 2,9 / 2,6 / 2,4 (wrong order)
-- NLJ: 2,4 / 2,6 / 2,9 (correct)
```
The plans differ: PWMJ sorts only on `l.v` (`SortExec: expr=[v@0 ASC]`);
NLJ sorts on both (`[v@0 ASC, v@1 ASC]`).
## What changes are included in this PR?
- `compute_properties` no longer passes the range `on` pair to
`join_equivalence_properties` (a range join adds no column
equivalences). The now-unused `join_on` parameter is dropped.
- Existing `pwmj.slt` plans/results updated to the corrected (fully
sorted) output — they previously encoded the wrong ordering — and a
regression test is added.
## Are these changes tested?
Yes.
- New regression test in `pwmj.slt` (INNER `l.v < r.v`, `WHERE l.v = 2`,
`ORDER BY r.v`) asserting the correctly ordered result.
- The existing `pwmj.slt` cases that asserted the wrong ordering / a
single-column sort plan are updated to the correct values (verified
against `NestedLoopJoin`).
- `piecewise` unit tests and `joins.slt` still pass. Only `INNER` is
affected.
## Are there any user-facing changes?
`INNER` range joins via `PiecewiseMergeJoin` (behind
`enable_piecewise_merge_join`, default off) no longer drop a required
sort, matching `NestedLoopJoin`. No API changes.1 parent 7c079f7 commit cc7565b
2 files changed
Lines changed: 43 additions & 12 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
363 | 363 | | |
364 | 364 | | |
365 | 365 | | |
366 | | - | |
367 | 366 | | |
368 | 367 | | |
369 | 368 | | |
| |||
425 | 424 | | |
426 | 425 | | |
427 | 426 | | |
428 | | - | |
429 | 427 | | |
430 | 428 | | |
431 | 429 | | |
| |||
434 | 432 | | |
435 | 433 | | |
436 | 434 | | |
437 | | - | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
438 | 441 | | |
439 | 442 | | |
440 | 443 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
129 | 129 | | |
130 | 130 | | |
131 | 131 | | |
132 | | - | |
| 132 | + | |
133 | 133 | | |
134 | 134 | | |
135 | 135 | | |
| |||
146 | 146 | | |
147 | 147 | | |
148 | 148 | | |
149 | | - | |
150 | 149 | | |
151 | | - | |
| 150 | + | |
152 | 151 | | |
153 | | - | |
| 152 | + | |
154 | 153 | | |
| 154 | + | |
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| |||
174 | 174 | | |
175 | 175 | | |
176 | 176 | | |
177 | | - | |
| 177 | + | |
178 | 178 | | |
179 | 179 | | |
180 | 180 | | |
| |||
238 | 238 | | |
239 | 239 | | |
240 | 240 | | |
241 | | - | |
242 | | - | |
243 | 241 | | |
244 | | - | |
| 242 | + | |
| 243 | + | |
245 | 244 | | |
| 245 | + | |
246 | 246 | | |
247 | 247 | | |
248 | 248 | | |
| |||
266 | 266 | | |
267 | 267 | | |
268 | 268 | | |
269 | | - | |
| 269 | + | |
270 | 270 | | |
271 | 271 | | |
272 | 272 | | |
| |||
387 | 387 | | |
388 | 388 | | |
389 | 389 | | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
390 | 418 | | |
391 | 419 | | |
392 | 420 | | |
| |||
0 commit comments