Skip to content

Commit 348553b

Browse files
authored
fix: support zero-field struct group keys in aggregate emit (apache#25282)
## Which issue does this PR close? Closes apache#25281. ## Rationale for this change `GROUP BY` and `SELECT DISTINCT` over a group key whose type contains a zero-field struct (`Struct()`) panicked when the aggregate emitted: ```sql select arrow_cast(null, 'Struct()') as s, count(*) from (values (1),(2),(3)) group by 1; -- thread 'tokio-rt-worker' panicked ... dictionary re-encode during emit: -- InvalidArgumentError("use StructArray::try_new_with_length or StructArray::new_empty_fields ...") ``` The same panic hit `SELECT DISTINCT` and keys where the empty struct is nested inside another struct. A zero-field struct is a valid Arrow type and should group like any other key. ## What changes are included in this PR? `encode_array_if_necessary` in `group_values/row.rs` rebuilt every struct column with `StructArray::try_new`, which arrow rejects for zero fields since there is no child column to derive the length from. It now uses `StructArray::try_new_with_length` with the source array's length, so zero-field structs are rebuilt like any other struct. This function is shared by `GroupValuesRows` and the row-backed `GroupColumn`, so both paths are covered. ## What is the testing strategy for this PR? Three new `sqllogictest` cases at the end of `struct.slt` (whole-key, `DISTINCT`, and nested inside `named_struct`). All three panicked before the fix and pass after it. ## Are there any user-facing changes? No, other than the panic being fixed. Signed-off-by: Fredrik Fornwall <[email protected]>
1 parent f4ca8a8 commit 348553b

2 files changed

Lines changed: 21 additions & 1 deletion

File tree

  • datafusion
    • physical-plan/src/aggregates/group_values
    • sqllogictest/test_files

‎datafusion/physical-plan/src/aggregates/group_values/row.rs‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -316,10 +316,12 @@ pub(crate) fn encode_array_if_necessary(
316316
})
317317
.collect::<Result<Vec<_>>>()?;
318318

319-
Ok(Arc::new(StructArray::try_new(
319+
// A zero-field struct has no child to infer the length from.
320+
Ok(Arc::new(StructArray::try_new_with_length(
320321
expected_fields.clone(),
321322
arrays,
322323
struct_array.nulls().cloned(),
324+
struct_array.len(),
323325
)?))
324326
}
325327
(DataType::List(expected_field), &DataType::List(_)) => {

‎datafusion/sqllogictest/test_files/struct.slt‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1872,3 +1872,21 @@ select a, s['b'] from (select -a as a, s from leaf_same_name order by a);
18721872

18731873
statement ok
18741874
drop table leaf_same_name;
1875+
1876+
# GROUP BY / DISTINCT over a zero-field struct key must not panic on emit
1877+
query ?I
1878+
select arrow_cast(null, 'Struct()') as s, count(*) from (values (1),(2),(3)) group by 1;
1879+
----
1880+
NULL 3
1881+
1882+
query ?
1883+
select distinct arrow_cast(null, 'Struct()') as s from (values (1),(2));
1884+
----
1885+
NULL
1886+
1887+
query ?I rowsort
1888+
select named_struct('a', column1, 'z', arrow_cast(null, 'Struct()')) as s, count(*)
1889+
from (values (1),(2),(1)) group by 1;
1890+
----
1891+
{a: 1, z: NULL} 2
1892+
{a: 2, z: NULL} 1

0 commit comments

Comments
 (0)