Repository navigation
Left/full outer join incorrect for CollectLeft / broadcast #1055
Description
Describe the bug
See discussion here apache/datafusion#12454
The "broadcast join" (CollectLeft) is wrong for certain join types which produce results on unmatched left rows.
To Reproduce
Run a broadcast join with left / full outer on more than one node
Expected behavior
Additional context
Activity
[-]Left/full outer join incorrect [/-][+]Left/full outer join incorrect for CollectLeft / broadcast[/+]Dandandan commentedon Nov 20, 2024
There is a proposal in DataFusion for adding a hook to support sharing the join state apache/datafusion#12523
We tested this at Coralogix, this works very well for us.
Dandandan commentedon Nov 20, 2024
It could be disabled as well, although that will likely hurt performance by quite a bit.
milenkovicm commentedon Nov 20, 2024
should we take this once it gets merged in DF ?
fix: Disable CollectLeft as it is broken in ballista
feat: Pinot-style colocated-join optimizer for hash-bucketed tables
feat: Pinot-style colocated-join optimizer for hash-bucketed tables
feat: Pinot-style colocated-join optimizer for hash-bucketed tables
milenkovicm commentedon May 24, 2026
#1647 fixed this issue
Dandandan commentedon Jun 28, 2026
I don't fully understand how #1647 would fix the issue? 🤔
andygrove commentedon Jun 28, 2026
It looks like this issue was closed prematurely. The PR that was merged does state that "AQE is deferred". Let's reopen this issue.
milenkovicm commentedon Jun 29, 2026
I apologise, I mistakenly thought this has been fixated