Repository navigation
Commit 32f39b4
authored
fix(physical-plan): report Final emission for LeftSemi hash and neste… (apache#24976)
## Which issue does this PR close?
- apache#24962
## Rationale for this change
`HashJoinExec` and `NestedLoopJoinExec` report `LeftSemi` joins as
emitting incrementally, but they only emit matched build-side rows after
the probe
side is exhausted.
Reporting `EmissionType::Final` reflects their actual behavior and
allows `SanityCheckPlan` to reject pipelines with an unbounded probe
side that cannot produce output.
## What changes are included in this PR?
- Classify `LeftSemi` joins as `EmissionType::Final` in both operators,
regardless of the probe input's emission type.
- Added unit tests verifying the emission type of LeftSemi joins in both
`HashJoinExec` and `NestedLoopJoinExec`.
- Add an integration test covering input swapping for an unbounded left
input, rejection of an unbounded probe input, and successful planning
when
both inputs are bounded.
## Are these changes tested?
Added unit tests verifying the emission type of LeftSemi joins in both
`HashJoinExec` and `NestedLoopJoinExec`.
Added an integration test covering:
- Unbounded left and bounded right: planning succeeds after swapping to
RightSemi.
- Bounded left and unbounded right: planning is rejected.
- Both inputs bounded: planning succeeds.
Following test commands have been executed and passed
- `cargo test -p datafusion`
- `cargo test --profile=ci --test sqllogictests`
- `cargo test -p datafusion`
- `cargo test -p datafusion-cli`
## Are there any user-facing changes?
Yes.
`LeftSemi` hash and nested loop joins now report Final emission.
Plans with a bounded build side and an unbounded probe side are rejected
instead of being accepted despite being unable to produce output.
Results for bounded inputs are unchanged.1 parent a01dc70 commit 32f39b4
3 files changed
Lines changed: 116 additions & 34 deletions
File tree
- datafusion
- core/tests/physical_optimizer
- physical-plan/src/joins
- hash_join
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
186 | 186 | | |
187 | 187 | | |
188 | 188 | | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
189 | 228 | | |
190 | 229 | | |
191 | 230 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1218 | 1218 | | |
1219 | 1219 | | |
1220 | 1220 | | |
1221 | | - | |
1222 | | - | |
1223 | | - | |
1224 | | - | |
1225 | | - | |
1226 | | - | |
1227 | | - | |
| 1221 | + | |
| 1222 | + | |
| 1223 | + | |
| 1224 | + | |
| 1225 | + | |
| 1226 | + | |
| 1227 | + | |
| 1228 | + | |
| 1229 | + | |
| 1230 | + | |
| 1231 | + | |
| 1232 | + | |
| 1233 | + | |
1228 | 1234 | | |
1229 | | - | |
1230 | | - | |
1231 | | - | |
1232 | | - | |
1233 | | - | |
| 1235 | + | |
| 1236 | + | |
1234 | 1237 | | |
1235 | 1238 | | |
1236 | 1239 | | |
| |||
4732 | 4735 | | |
4733 | 4736 | | |
4734 | 4737 | | |
| 4738 | + | |
| 4739 | + | |
| 4740 | + | |
| 4741 | + | |
| 4742 | + | |
| 4743 | + | |
| 4744 | + | |
| 4745 | + | |
| 4746 | + | |
| 4747 | + | |
| 4748 | + | |
| 4749 | + | |
| 4750 | + | |
| 4751 | + | |
| 4752 | + | |
| 4753 | + | |
| 4754 | + | |
| 4755 | + | |
| 4756 | + | |
| 4757 | + | |
| 4758 | + | |
| 4759 | + | |
| 4760 | + | |
| 4761 | + | |
| 4762 | + | |
4735 | 4763 | | |
4736 | 4764 | | |
4737 | 4765 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
390 | 390 | | |
391 | 391 | | |
392 | 392 | | |
393 | | - | |
394 | | - | |
395 | | - | |
396 | | - | |
397 | | - | |
398 | | - | |
399 | | - | |
400 | | - | |
401 | | - | |
402 | | - | |
403 | | - | |
404 | | - | |
405 | | - | |
406 | | - | |
407 | | - | |
408 | | - | |
409 | | - | |
410 | | - | |
411 | | - | |
412 | | - | |
413 | | - | |
414 | | - | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
415 | 418 | | |
416 | 419 | | |
417 | 420 | | |
| |||
4954 | 4957 | | |
4955 | 4958 | | |
4956 | 4959 | | |
| 4960 | + | |
| 4961 | + | |
| 4962 | + | |
| 4963 | + | |
| 4964 | + | |
| 4965 | + | |
| 4966 | + | |
| 4967 | + | |
| 4968 | + | |
| 4969 | + | |
| 4970 | + | |
| 4971 | + | |
4957 | 4972 | | |
4958 | 4973 | | |
4959 | 4974 | | |
| |||
0 commit comments