arrow-row: expose decoded_type API so downstreams can compute row-format round-trip type expectations #10427

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

RowConverter accepts types on encode whose decode output type differs from the encode input type: dictionaries (and dictionary-containing nested types) are flattened to their value type on decode via the internal corrected_type step. For example:

• Dictionary(Int32, Utf8) decodes as Utf8
• ListView(Dictionary(Int32, Utf8)) decodes as ListView(Utf8)
• List(Dictionary(Int32, Utf8)) decodes as List(Utf8)

This mapping is arrow-row internal knowledge with no public API to query it. Downstreams that need to round-trip arrays through the row format (e.g. DataFusion's group-by, which stores group keys as rows and re-emits them as arrays) must mirror decoder internals by hand to know which output types to expect and reconstruct.

This has produced two real bugs of the same class:

1. arrow-row: decode_fixed_size_list panics on FixedSizeList<Dictionary> (missing corrected_type) #10413 — decode_fixed_size_list diverged from the other list-like decoders by not applying the correction, panicking on FixedSizeList<Dictionary> (fixed by arrow-row: Fix decode_fixed_size_list to apply the corrected_type step for dictionary children #10414).
2. apache/datafusion#23523 review — DataFusion accepted ListView<Dictionary> for its rows-backed group storage but missed the reconstruction arm for view-lists, silently emitting ListView(Utf8) where the schema declared ListView(Dictionary(Int32, Utf8)).

Both stem from the same gap: each downstream has to re-derive, per container type, "what type will decode hand me back?" — and every missed container is a bug.

Describe the solution you'd like

Expose the mapping as a public API, e.g.:

/// Returns the data type produced by `RowConverter::convert_rows` for
/// an input column of `data_type` — identical to the input except that
/// dictionary-encoded values (at any nesting level) are replaced by
/// their value type.
pub fn decoded_type(data_type: &DataType) -> DataType

(placement flexible: free function in arrow_row, method on SortField, or RowConverter::decoded_types(&self) -> Vec<DataType>).

Together with a property test in arrow-row asserting the contract for every supported type shape:

for dt in representative_supported_shapes() {
    let array = make_test_array(&dt);
    let rows = converter.convert_columns(&[array])?;
    let decoded = converter.convert_rows(&rows)?;
    assert_eq!(decoded[0].data_type(), &decoded_type(&dt));
}

The invariant test is arguably the bigger win: it would have caught #10413 automatically (panic = contract violation), and prevents any future decoder from silently diverging from the others the way decode_fixed_size_list did.

With this API, downstream code reduces to a mechanical rule: decoded_type(t) == t → array round-trips as-is; otherwise recurse over the structure and re-encode exactly where the two types differ.

Describe alternatives you've considered

• A boolean round_trips_losslessly(&DataType) -> bool — simpler, but only says whether the type changes, not what comes back, so downstreams still can't derive reconstruction mechanically.
• Making decode preserve dictionary types — would eliminate the mapping entirely, but re-encoding dictionaries on decode is expensive, changes long-standing semantics, and the flattened output is often what callers want.
• Status quo — downstreams mirror decoder internals; two bugs in two months suggests this doesn't scale.

Additional context

• arrow-row: decode_fixed_size_list panics on FixedSizeList<Dictionary> (missing corrected_type) #10413 / arrow-row: Fix decode_fixed_size_list to apply the corrected_type step for dictionary children #10414 (the decode_fixed_size_list divergence)
• apache/datafusion#23523 — the DataFusion PR where the second instance surfaced, with an end-to-end reproducer in the review thread

Happy to implement this (API + property test) if the direction sounds good.