Apache Iceberg Rust version

None

Describe the bug

Consider the following single-column table of nullable strings:

a
-----
null
"foo"
"bar"

When querying using the predicate a IS NULL OR a = "foo", You'd expect the result to be null and "foo".

Similarly, when querying using the predicate a IS NOT NULL AND a != "foo", You'd expect the result to be just "bar".

Right now, in the first instance we get the result ["foo"], and in the second we get ["foo", "bar"].

The issue is caused by Iceberg AND and OR predicates being mapped to arrow_arith's and and or predicates. I think we should be transforming Iceberg AND and OR predicates into and_kleene and or_kleene.

See here: https://github.com/apache/iceberg-rust/blob/main/crates%2Ficeberg%2Fsrc%2Farrow%2Freader.rs#L822-L844

See https://docs.rs/arrow-arith/latest/arrow_arith/boolean/fn.or.html vs https://docs.rs/arrow-arith/latest/arrow_arith/boolean/fn.or_kleene.html

To Reproduce

See #1045

Expected behavior

No response

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time