FanoutWriter and ClusteredWriter merge -0.0 and 0.0 identity partitions of float and double columns #3325

Apache Iceberg Rust version

main. Verified at bb1e4a4 (2026-09-21); the code referenced below is unchanged on main today.

Describe the bug

FanoutWriter and ClusteredWriter put rows whose float/double identity partition value is -0.0 and rows whose value is 0.0 into the same data file, under whichever value arrived first. iceberg-java writes two partitions, because it compares partition keys with Float.compare / Double.compare, which keep the two apart.

Both writers group rows by the partition Struct:

• FanoutWriter keeps partition_writers: HashMap<Struct, _>
• ClusteredWriter keeps current_partition: Option<Struct> and closed_partitions: HashSet<Struct>

PrimitiveLiteral derives PartialEq, Eq and Hash over OrderedFloat, and OrderedFloat treats -0.0 and 0.0 as equal with the same hash. So the second value finds the first value's writer.

The data file is then recorded with one partition value while holding rows for both, so a scan that prunes on the other value drops rows. Seen through Spark on a table (id INT, d DOUBLE) partitioned by d, after writing (1, -0.0) and (2, 0.0) in one task:

Related work that does not cover this:

• Float comparison does not follow IEEE 754 totalOrder for -0.0 vs +0.0 #1951 / fix: follow IEEE 754 totalOrder for float and double #1959 made Datum ordering follow IEEE 754 totalOrder. That changed ordering only; equality and hashing of partition structs are unchanged.
• fix(spec): preserve exact schema and temporal value semantics #3077 adds a bit-exact same_value on PrimitiveLiteral for comparing schema defaults. It does not change the derived Eq / Hash the writers use.

To Reproduce

use std::collections::HashMap;

use iceberg::spec::{Literal, Struct};

let neg = Struct::from_iter([Some(Literal::double(-0.0))]);
let pos = Struct::from_iter([Some(Literal::double(0.0))]);

// Passes: the two partition values are equal.
assert_eq!(neg, pos);

// Passes: looking up 0.0 returns the writer opened for -0.0, as in `FanoutWriter`.
let mut partition_writers: HashMap<Struct, &str> = HashMap::new();
partition_writers.insert(neg, "writer for -0.0");
assert_eq!(partition_writers.get(&pos), Some(&"writer for -0.0"));

The same holds for Literal::float.

Expected behavior

-0.0 and 0.0 are distinct partition values, as in iceberg-java, so the writers produce two data files with two partition values.

Possible directions: key the writers on a bit-exact form of the partition struct, or make equality and hashing of float literals bit-exact (one canonical NaN, signed zeros distinct), which is the Java Float.equals / Double.equals behavior that same_value in #3077 implements.

Willingness to contribute

None

Additional context

Found through apache/datafusion-comet#6138. Comet's native Iceberg writer uses these writers, and is adding a fall-back for float/double identity partitions until this is resolved.