You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Null-aware NOT IN hash join in Partitioned mode returns rows when the subquery contains NULL #26496
An uncorrelated null-aware anti join (build.c1 NOT IN (probe.c2), JoinType::LeftAnti with null_aware = true) built with PartitionMode::Partitioned returns rows even when the probe side contains a NULL. SQL semantics say that if the subquery produces a NULL, x NOT IN (...) is never true, so the result must be empty. The same join in PartitionMode::CollectLeft returns the correct empty result.
The cause is that in Partitioned mode every output partition builds its own build side, and the "probe side saw a NULL" fact is recorded on that per-partition build data (hash_join/exec.rs around line 1935, hash_join/stream.rs around line 1366). A NULL probe key is hash-routed to one partition, so it only suppresses the build rows of that partition. The others still emit their unmatched rows.
NullAwareMode::try_new already rejects Partitioned for the correlated LeftAnti and LeftMark forms, but accepts it for the uncorrelated LeftAnti form.
To Reproduce
The DataFusion SQL planner never produces this plan: JoinSelection and the physical planner both pin null-aware joins to CollectLeft. It is reachable from plans built by hand or deserialized from protobuf (HashJoinExecNode decodes the partition mode as written).
Build side c1 = 1..8, probe side c2 = {1, NULL}, both sides RepartitionExec(Hash([c], 4)), HashJoinExec with JoinType::LeftAnti, PartitionMode::Partitioned, null_aware = true, NullEqualsNothing.
Expected behavior
No rows (as CollectLeft returns).
Actual behavior
[3, 6, 8] — the build rows hashed to partitions that did not receive the NULL probe row.
Additional context
A fix that rejects this shape at plan construction, consistent with the existing rejection for the correlated forms, is ready; I'll link the PR.
Activity
cool9850311 commentedon Oct 19, 2026
take
fix: reject null-aware `LeftAnti` hash join in `Partitioned` mode
fix: reject null-aware `LeftAnti` hash join in `Partitioned` mode (ap…