PrimitiveArray's Debug impl generates ~85K lines of LLVM IR in every downstream crate that debug-formats arrays #10889

Component(s): arrow-array

Is your feature request related to a problem or challenge?

While surveying the DataFusion workspace for binary-size opportunities with cargo llvm-lines

• [EPIC] Reduce binary size datafusion#24727

The single largest generic-code item in every crate I measured was not DataFusion code at all — it was the Debug impl for PrimitiveArray<T>:

Specifically:

That is ~85,000 IR lines (~4-6% of each crate's total codegen), and because generic functions are instantiated in the consuming crate, the same ~85K lines are generated again in every crate that debug-formats a PrimitiveArray — directly or indirectly. Debug is reachable from unwrap/expect/assert_eq! and error formatting, so essentially every crate that touches Arrow arrays pays this.

What is going on

The Debug impl is generic over T: ArrowPrimitiveType and is monomorphized for all ~32 primitive types:

https://github.com/apache/arrow-rs/blob/59.2.0/arrow-array/src/array/primitive_array.rs#L1342-L1412

impl<T: ArrowPrimitiveType> std::fmt::Debug for PrimitiveArray<T> {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        let data_type = self.data_type();

        write!(f, "PrimitiveArray<{data_type}>\n[\n")?;
        print_long_array(self, f, |array, index, f| match data_type {
            DataType::Date32 | DataType::Date64 => { /* as_date::<T> ... */ }
            DataType::Time32(_) | DataType::Time64(_) => { /* as_time::<T> ... */ }
            DataType::Timestamp(_, tz_string_opt) => {
                /* timezone parsing, as_datetime_with_timezone::<T>, RFC3339 formatting,
                   error messages for invalid timezones ... */
            }
            _ => std::fmt::Debug::fmt(&array.value(index), f),
        })?;
        write!(f, "]")
    }
}

The per-value closure contains the entire temporal-formatting match — date/time conversion, timezone parsing, RFC3339 formatting — and because the match is on the runtime value self.data_type(), the compiler cannot prune those arms even for types like Int8Type that can never be temporal. Every one of the 32 instantiations carries all of it.

In addition, print_long_array is generic over both the array type and the closure type, so the head/tail truncation logic is also duplicated 32 times:

https://github.com/apache/arrow-rs/blob/59.2.0/arrow-array/src/array/mod.rs#L1045-L1080

The only genuinely type-specific code in all of this is the default arm: Debug::fmt(&array.value(index), f) for T::Native — a handful of lines. Everything else is identical across instantiations.

Describe the solution you'd like

Restructure so the shared machinery is compiled once, keeping only the tiny per-native-type value formatting monomorphized. For example:

• Change print_long_array to take &dyn Array and &mut dyn FnMut(...) instead of being generic over A and F (it is called through Array trait methods only), and/or
• Move the temporal match out of the per-value closure into a non-generic helper: select a formatting function by matching data_type once, then have the generic fmt pass only the trivial Debug::fmt(&value) closure for the default case.

Debug formatting is not performance-sensitive, so the usual monomorphization-for-speed argument does not apply; this should be pure code-size win with no measurable cost. A similar runtime-dispatch restructuring in DataFusion (apache/datafusion#24687) removed 67% of a module's generated code with benchmark-verified zero runtime impact.

The same print_long_array pattern is used by other Debug impls (GenericByteArray, GenericListArray, etc.); those have far fewer instantiations, but would benefit from the same non-generic helper.

Describe alternatives you've considered

Leaving it as is: downstream binaries pay a few hundred KB and every crate compiling against arrow-array spends LLVM time on ~85K lines of duplicate IR.

Additional context

Numbers above are from cargo llvm-lines --release -p <crate> --lib (cargo-llvm-lines 0.4.41) on DataFusion crates using arrow-array 59.2.0. Reproduce with:

cargo install cargo-llvm-lines
cargo llvm-lines --release -p datafusion-physical-expr --lib | grep -E "print_long_array|PrimitiveArray.*Debug"

• Found while working on reducing DataFusion binary size: [EPIC] Reduce binary size datafusion#24727