You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Null-aware LeftMark joins miss logical NULLs in dictionary keys #25691
Null-aware LeftMark hash joins inspect the physical null bitmap of dictionary-encoded join keys instead of their logical nulls. A DictionaryArray key can be physically valid while pointing to a NULL dictionary value, so the join misses that NULL and produces an incorrect FALSE mark where SQL three-valued logic requires NULL.
The problem affects both sides of an uncorrelated null-aware mark join:
A logical NULL build key is treated as non-null when the final mark column is built.
A logical NULL probe key is not recorded in the shared probe summary, including when it is seen by a different probe partition from the partition that completes the join.
To Reproduce
At the HashJoinExec level, use dictionary keys whose key bitmap contains no nulls but whose dictionary values contain a null:
Build values [1, NULL, 4], probe values [1, 2, 3]: the expected marks are [TRUE, NULL, FALSE], but the build-side logical null is marked FALSE.
Build values [1, 4], probe values containing [1, NULL]: the expected marks are [TRUE, NULL], but the unmatched build row is marked FALSE.
Expected behavior
Null-aware LeftMark joins should use Arrow logical-null semantics for dictionary arrays on both build and probe sides, producing the same SQL results as non-dictionary arrays.
Activity
fix: handle dictionary nulls in null-aware left mark joins (apache#25693
fix: handle dictionary nulls in null-aware left mark joins (apache#25693