Skip to content

Commit 8d58672

Browse files
committed
Give the results of the storage pushdown the extension dtype
`ExtensionStoragePushdown` returned the results of its aggregate over storage values unchanged. So `vortex.min_max[vortex.timestamp]` gave raw `i64` values without the time unit or zone, while plain `min_max` gives timestamps. The new `StorageResultMapping` trait names the value fields of a partial and a result, which hold input values, and `MinMax` and `IsSorted` implement it. The pushdown requires the trait, so an aggregate without a mapping does not compile. Partial states stay in storage form in memory. `to_scalar` and `finalize_scalar` give their output the extension dtype, `partial_from_scalar` removes it, and `finalize` receives states that already have it. The tests now compare the pushdown with plain `min_max` over flat, chunked, constant, dictionary-encoded, and grouped input, and check that an `is_sorted` partial with a null first value survives its round trip through a scalar. The test that registered the pushdown by hand is gone, because the default session test already checks the registration. Signed-off-by: Connor Tsui <[email protected]>
1 parent 8fea5fd commit 8d58672

4 files changed

Lines changed: 241 additions & 107 deletions

File tree

‎encodings/uuid/src/lib.rs‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -158,13 +158,16 @@ mod tests {
158158
let uuid = |leading_byte: u8| {
159159
let mut bytes = [0u8; 16];
160160
bytes[0] = leading_byte;
161-
Scalar::fixed_size_list(
162-
DType::Primitive(PType::U8, Nullability::NonNullable),
163-
bytes
164-
.into_iter()
165-
.map(|byte| Scalar::primitive(byte, Nullability::NonNullable))
166-
.collect(),
167-
Nullability::NonNullable,
161+
Scalar::extension::<Uuid>(
162+
UuidMetadata::default(),
163+
Scalar::fixed_size_list(
164+
DType::Primitive(PType::U8, Nullability::NonNullable),
165+
bytes
166+
.into_iter()
167+
.map(|byte| Scalar::primitive(byte, Nullability::NonNullable))
168+
.collect(),
169+
Nullability::NonNullable,
170+
),
168171
)
169172
};
170173
assert_eq!(

‎vortex-array/src/aggregate_fn/fns/is_sorted/mod.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ use crate::aggregate_fn::AggregateArgs;
3232
use crate::aggregate_fn::AggregateFnId;
3333
use crate::aggregate_fn::AggregateFnVTable;
3434
use crate::aggregate_fn::DynAccumulator;
35+
use crate::aggregate_fn::StorageResultMapping;
3536
use crate::aggregate_fn::fns::fixed_size_bytes::is_fixed_size_bytes;
3637
use crate::arrays::Constant;
3738
use crate::arrays::Null;
@@ -582,6 +583,12 @@ impl AggregateFnVTable for IsSorted {
582583
}
583584
}
584585

586+
impl StorageResultMapping for IsSorted {
587+
// `first_value` and `last_value` are the value fields of the partial. The result is a `Bool`.
588+
const PARTIAL_VALUE_FIELDS: &'static [usize] = &[2, 3];
589+
const RETURN_VALUE_FIELDS: &'static [usize] = &[];
590+
}
591+
585592
#[expect(
586593
clippy::wrong_self_convention,
587594
reason = "is_* naming follows Iterator::is_sorted convention"

‎vortex-array/src/aggregate_fn/fns/min_max/mod.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ use crate::aggregate_fn::AggregateFnId;
3434
use crate::aggregate_fn::AggregateFnVTable;
3535
use crate::aggregate_fn::DynAccumulator;
3636
use crate::aggregate_fn::NumericalAggregateOpts;
37+
use crate::aggregate_fn::StorageResultMapping;
3738
use crate::aggregate_fn::fns::fixed_size_bytes::is_fixed_size_bytes;
3839
use crate::dtype::DType;
3940
use crate::dtype::FieldNames;
@@ -538,6 +539,12 @@ impl AggregateFnVTable for MinMax {
538539
}
539540
}
540541

542+
impl StorageResultMapping for MinMax {
543+
// `min` and `max` are the value fields of both the partial and the result.
544+
const PARTIAL_VALUE_FIELDS: &'static [usize] = &[0, 1];
545+
const RETURN_VALUE_FIELDS: &'static [usize] = &[0, 1];
546+
}
547+
541548
#[cfg(test)]
542549
mod tests {
543550
use std::sync::Arc;

0 commit comments

Comments
 (0)