Repository navigation
Spark xxhash64 ignores validity buffer #25505
Description
Describe the bug
To Reproduce
CREATE TABLE t AS SELECT named_struct('a', 1, 'b', 'x') AS s;
-- nullif only clears the struct's validity bit; the child arrays still hold 1 and 'x'
SELECT s IS NULL AS is_null,
xxhash64(s) AS masked_null,
xxhash64(arrow_cast(NULL, 'Struct("a": Int64, "b": Utf8)')) AS plain_null
FROM (SELECT nullif(s, s) AS s FROM t);$ cargo run --bin datafusion-cli -- --spark -f repro.sql
+---------+---------------------+------------+
| is_null | masked_null | plain_null |
+---------+---------------------+------------+
| true | 6158280145716684778 | 42 |
+---------+---------------------+------------+
Expected behavior
No response
Additional context
No response
Activity
fix: xxhash64 of a NULL struct ignores the struct fields (apache#25774)
fix: xxhash64 of a NULL struct ignores the struct fields (apache#25774)