Skip to content

Commit 597e2ae

Browse files
claudeconnortsui20
andcommitted
docs(aggregate): adopt the reviewed ExtensionStoragePushdown doc comment
Co-authored-by: connortsui20 <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Hp7ZGEG8mrip6qkAtK4iWD Signed-off-by: Claude <[email protected]>
1 parent 511a84a commit 597e2ae

1 file changed

Lines changed: 28 additions & 21 deletions

File tree

‎vortex-array/src/aggregate_fn/storage_pushdown.rs‎

Lines changed: 28 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -30,31 +30,38 @@ use crate::dtype::DType;
3030
use crate::dtype::extension::ExtVTable;
3131
use crate::scalar::Scalar;
3232

33-
/// An opt-in pushdown of aggregate `A` into the storage of extension type `E`.
33+
/// Runs aggregate `A` on the storage values of extension type `E`.
3434
///
35-
/// Registering a pair declares that `A` over `E`'s storage is correct for `E`. Nothing checks
36-
/// this, and a wrong registration silently returns wrong answers. Only register a pair when both
37-
/// of these hold:
35+
/// The input must have extension type `E`, and `A` must support its storage dtype. Binding fails
36+
/// for any other input. The wrapper has a separate aggregate ID, `<A>[<E>]`, formed from the IDs
37+
/// of `A` and `E`. Registering it does not replace `A`.
3838
///
39-
/// - **`E`'s domain supports the aggregation.** The result must mean something for values of `E`.
40-
/// The minimum of some timestamps is an instant, but their sum is not.
41-
/// - **The storage computes it soundly.** Every operation `A` uses must give the same answer on
42-
/// the storage as on `E`:
43-
/// - comparison (`min`, `max`, `min_max`, `bounded_*`, `is_sorted`): `E` orders like its
44-
/// storage.
45-
/// - equality (`is_constant`, `all_non_distinct`): each value of `E` has exactly one storage
46-
/// value.
47-
/// - addition (`sum`, `mean`): adding storage values gives the storage of their sum.
48-
/// - NaN detection (`nan_count`, `all_nan`, `all_non_nan`): a NaN in the storage is a NaN of `E`.
39+
/// Aggregates that already support extension dtypes directly do not need this wrapper.
4940
///
50-
/// For example, `Date`, `Time` and `Timestamp` pass for comparison but not for `sum`. Aggregates
51-
/// that already accept any extension dtype, such as `count` and `null_count`, need no pushdown.
41+
/// # Correctness
5242
///
53-
/// Inputs must be extension dtypes of type `E` whose storage `A` accepts; anything else fails to
54-
/// bind. Results are `A`'s over the storage and are not converted back to `E`, so the extrema of a
55-
/// timestamp column are raw `i64` values in the column's unit. Partials are only comparable for
56-
/// the same extension dtype, including metadata such as a timestamp's unit. The pushdown is an
57-
/// aggregate of its own, with id `<A>[<E>]`, and never replaces `A`.
43+
/// A registration **must** satisfy both conditions:
44+
///
45+
/// - The aggregation is defined for logical values of `E`.
46+
/// - For every valid input, aggregating the storage values produces the storage representation
47+
/// of the logical aggregate result. This requires storage to preserve whichever properties `A`
48+
/// relies on, such as ordering, equality, arithmetic, or NaN classification.
49+
///
50+
/// For example, minimum is defined for timestamps, and their integer storage preserves their
51+
/// ordering, so both conditions hold. Summing timestamps fails the first condition even though
52+
/// their storage supports addition.
53+
///
54+
/// These conditions are not checked. An invalid registration can silently return incorrect results.
55+
///
56+
/// # Results and partials
57+
///
58+
/// Results and partial states are those of `A` over the storage dtype. Results are not converted
59+
/// back to `E`. For example, the minimum and maximum of a timestamp column are raw `i64` values
60+
/// in the column's time unit.
61+
///
62+
/// Partial states can only be combined when the aggregate, options, and full input dtype match.
63+
/// This includes extension metadata, so timestamp partials with different time units must not be
64+
/// combined even if their storage dtypes match.
5865
///
5966
/// # Example
6067
///

0 commit comments

Comments
 (0)