-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Refactor decimal casting in arrow-cast to converge on the same rescale logic used by the parquet variant cast
Describe the solution you'd like
Originally post by @scovich in #8552 (comment)
rework
convert_to_smaller_scale_decimalandconvert_to_bigger_or_equal_scale_decimal
- no longer take
arrayorcast_optionsas input- return
Ok((f, is_infallible_cast)which corresponds to the return typeResult<(impl Fn(I::Native) -> Option<O::Native>, bool), ArrowError>define a new generic
apply_decimal_castfunction helper
- it takes as input
array,cast_optionsand the(impl Fn, bool)pair produced by aconvert_to_xxx_scale_decimalhelper- it handles the three ways of applying
fto an arrayrework
cast_decimal_to_decimalandcast_decimal_to_decimal_same_typeto call those functions (see below)
rescale_decimalwould be the single-row equivalent ofcast_decimal_to_decimal, returningOption<O::Native>The decimal builder's constructor calls
validate_decimal_precision_and_scaleand fails on error, so we don't need to validate on a per-row basis.cast_decimal_to_decimal
let array: PrimitiveArray<O> = if input_scale > output_scale { let (f, is_infallible_cast) = convert_to_smaller_scale_decimal(...)?; apply_decimal_cast(array, cast_options, f, is_infallible)? } else { let (f, is_infallible_cast) = convert_to_bigger_or_equal_scale_decimal(...)?; apply_decimal_cast(array, cast_options, f, is_infallible)? }rescale_decimal
if input_scale > output_scale { let (f, _) = convert_to_smaller_scale_decimal(...)?; f(integer) } else { let (f, _) = convert_to_bigger_or_equal_scale_decimal(...)?; f(integer) }
Describe alternatives you've considered
Additional context