Skip to content

Commit f3ca34d

Browse files
committed
Correct why an output label must be an extension dtype
The previous wording derived the restriction from value preservation, which does not hold. An extension type can constrain its storage values through `ExtVTable::validate_scalar_value`, and `DivisibleInt` does. The restriction is structural. An extension array holds its storage column as a child, so a label applies by wrapping a finished column of any encoding. No other dtype has that form, so labelling to one would be a cast rather than a wrap. Also records that labelling compares dtypes and does not validate values, which is the same trust the framework already places in `OutputElement::element_dtype`. Signed-off-by: Connor Tsui <[email protected]>
1 parent 0ab50e3 commit f3ca34d

3 files changed

Lines changed: 22 additions & 11 deletions

File tree

‎vortex-array/src/scalar_fn/unstable/row/batch/tests.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -549,7 +549,7 @@ fn test_declared_output_dtype_reaches_planning() -> VortexResult<()> {
549549

550550
#[rstest]
551551
#[case::nullable(timestamp_dtype(Nullability::Nullable), "must be non-nullable")]
552-
#[case::not_an_extension(DType::from(u64::PTYPE), "must label the storage dtype")]
552+
#[case::not_an_extension(DType::from(u64::PTYPE), "must be an extension dtype")]
553553
fn test_declared_output_dtype_rejects_bad_label(
554554
#[case] declared: DType,
555555
#[case] expected_message: &str,

‎vortex-array/src/scalar_fn/unstable/row/visitor/plan.rs‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -236,12 +236,15 @@ impl BatchPlan {
236236

237237
/// Validate a declared output dtype and return the label to apply to the storage column.
238238
///
239-
/// A declared dtype **must** be non-nullable and **must** leave every value unchanged, which
240-
/// restricts it to `storage_dtype` itself or to an extension dtype storing exactly that dtype.
241-
/// Returning `None` for the former keeps [`BatchPlan::relabel_output`] a no-op for it.
239+
/// A declared dtype **must** be non-nullable and **must** apply by wrapping the finished column,
240+
/// which restricts it to `storage_dtype` itself or to an extension dtype storing exactly that
241+
/// dtype. An extension array holds its storage column as a child, so wrapping costs nothing and
242+
/// works for any encoding. Returning `None` for the former keeps [`BatchPlan::relabel_output`] a
243+
/// no-op for it.
242244
///
243245
/// This is the single validation point for that relationship, which
244-
/// [`BatchPlan::relabel_output`] relies on.
246+
/// [`BatchPlan::relabel_output`] relies on. It compares dtypes only. An extension type that
247+
/// constrains its storage values trusts the row kernel to produce values that satisfy it.
245248
fn validate_output_label(
246249
storage_dtype: &DType,
247250
output_dtype: DType,
@@ -257,8 +260,8 @@ fn validate_output_label(
257260

258261
let DType::Extension(output_label) = output_dtype else {
259262
vortex_bail!(
260-
"a declared row output dtype must label the storage dtype {storage_dtype} without \
261-
changing any value, got {output_dtype}",
263+
"a declared row output dtype must be an extension dtype over the storage dtype \
264+
{storage_dtype}, got {output_dtype}",
262265
);
263266
};
264267
vortex_ensure_eq!(

‎vortex-array/src/scalar_fn/unstable/row/visitor/row_visitor.rs‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,10 +46,16 @@ pub trait RowVisitor: private::Sealed + Sized {
4646
/// Batch execution applies the label after deriving nullability and masking null rows, so an
4747
/// empty or all-null batch carries the same metadata as a populated one.
4848
///
49-
/// `dtype` **must** be non-nullable, and **must** label the storage dtype without changing any
50-
/// value. That restricts it to the storage dtype itself or to an extension dtype storing it.
51-
/// The visit validates both, and execution checks that this dispatch declares the dtype
52-
/// planning selected.
49+
/// `dtype` **must** be non-nullable, and **must** apply by wrapping the finished column. That
50+
/// restricts it to the storage dtype itself or to an extension dtype storing it, because an
51+
/// extension array holds its storage column as a child whatever that column's encoding. The
52+
/// visit validates both, and execution checks that this dispatch declares the dtype planning
53+
/// selected.
54+
///
55+
/// Labelling validates dtypes and not values. An extension type can constrain its storage
56+
/// values through [`ExtVTable::validate_scalar_value`], so a dispatch that declares one
57+
/// **must** produce values that satisfy it. This is the same trust the framework places in
58+
/// [`OutputElement::element_dtype`].
5359
///
5460
/// # Examples
5561
///
@@ -66,6 +72,8 @@ pub trait RowVisitor: private::Sealed + Sized {
6672
/// ))
6773
/// .visit::<(TimestampRow,), i64>(move |(value,)| value - value.rem_euclid(ticks))
6874
/// ```
75+
/// [`ExtVTable::validate_scalar_value`]: crate::dtype::extension::ExtVTable::validate_scalar_value
76+
/// [`OutputElement::element_dtype`]: crate::scalar_fn::unstable::row::OutputElement::element_dtype
6977
fn with_output_dtype(self, dtype: DType) -> Self;
7078

7179
/// Visit an infallible row computation that returns one output value per row.

0 commit comments

Comments
 (0)