Skip to content

Commit 1b28996

Browse files
authored
perf(array): hoist decimal same-scale cast plan (#8932)
## Summary - Evaluate the `SameScale` cast plan once for each array. - Use a direct closure for each value in a same-scale cast. - Use `cast_decimal_buffer` for dense, null, and masked output buffers. - Keep the existing operation for all other cast plans. ## Problem The previous code called `DecimalCastPlan::cast` for each value. That function matched the cast plan for each value. The benchmark profile uses 16 code-generation units and no LTO. An unrelated source addition changed the LLVM output for this function. The nullable copy benchmark increased from approximately 65 µs to 99 µs. Samply showed the same hot call path in both binaries: ```text try_map_masked_into -> DecimalCastPlan::cast ``` The slow binary used a larger function and more stack loads and stores. The new code moves the common plan match outside the value loop. This makes the loop independent of that match. ## Benchmarks Command: ```console cargo bench -p vortex-array --bench cast_decimal -- --sample-count 100 --min-time 3 --max-time 6 --color never ``` | benchmark | develop median | this PR median | change | |---|---:|---:|---:| | `copy_non_nullable[65536]` | 70.10 µs | 41.18 µs | -41.3% | | `copy_nullable[65536]` | 65.11 µs | 56.44 µs | -13.3% | | `in_place_non_nullable[65536]` | 590.5 ns | 610.7 ns | +20.2 ns | | `in_place_nullable[65536]` | 680.5 ns | 690.7 ns | +10.2 ns | The in-place cases do not use the changed buffer path. Their differences are 10 ns and 20 ns. ## Verification - `cargo nextest run -p vortex-array -E 'test(/arrays::decimal::compute::cast::tests/)'` — 23 passed - `cargo clippy --all-targets --all-features` - `cargo +nightly fmt --all --check` ## AI assistance This PR was prepared with OpenAI Codex. The human author reviewed the code, benchmark results, profile data, and test results. Signed-off-by: 蔡略 <[email protected]>
1 parent 640d802 commit 1b28996

1 file changed

Lines changed: 37 additions & 30 deletions

File tree

  • vortex-array/src/arrays/decimal/compute

‎vortex-array/src/arrays/decimal/compute/cast.rs‎

Lines changed: 37 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -216,41 +216,48 @@ where
216216
{
217217
let values = array.buffer::<F>();
218218
let values = values.as_slice();
219-
let cast_plan = DecimalCastPlan::<T>::new(from_decimal_dtype, to_decimal_dtype);
219+
let cast_result = match DecimalCastPlan::<T>::new(from_decimal_dtype, to_decimal_dtype) {
220+
DecimalCastPlan::SameScale { min, max } => {
221+
cast_decimal_buffer(values, valid_values, |value| {
222+
let value = <T as BigCast>::from(value)?;
223+
(value >= min && value <= max).then_some(value)
224+
})
225+
}
226+
cast_plan => cast_decimal_buffer(values, valid_values, |value| cast_plan.cast(value)),
227+
};
228+
let buffer = cast_result.map_err(|idx| {
229+
decimal_cast_error::<F, T>(values[idx], from_decimal_dtype, to_decimal_dtype)
230+
})?;
220231

221-
let buffer = match valid_values {
232+
Ok(DecimalArray::new(buffer, to_decimal_dtype, validity).into_array())
233+
}
234+
235+
fn cast_decimal_buffer<F, T>(
236+
values: &[F],
237+
valid_values: &Mask,
238+
mut cast: impl FnMut(F) -> Option<T>,
239+
) -> Result<Buffer<T>, usize>
240+
where
241+
F: NativeDecimalType,
242+
T: NativeDecimalType,
243+
{
244+
let mut buffer = BufferMut::<T>::with_capacity(values.len());
245+
match valid_values {
222246
Mask::AllTrue(_) => {
223-
let mut buffer = BufferMut::<T>::with_capacity(values.len());
224-
values
225-
.try_map_into(&mut buffer.spare_capacity_mut()[..values.len()], |value| {
226-
cast_plan.cast(value)
227-
})
228-
.map_err(|idx| {
229-
decimal_cast_error::<F, T>(values[idx], from_decimal_dtype, to_decimal_dtype)
230-
})?;
231-
// SAFETY: try_map_into initializes every lane before returning Ok.
232-
unsafe { buffer.set_len(values.len()) };
233-
buffer.freeze()
247+
values.try_map_into(&mut buffer.spare_capacity_mut()[..values.len()], &mut cast)?;
234248
}
235-
Mask::AllFalse(_) => BufferMut::<T>::zeroed(values.len()).freeze(),
249+
Mask::AllFalse(_) => return Ok(BufferMut::<T>::zeroed(values.len()).freeze()),
236250
Mask::Values(mask) => {
237-
let mut buffer = BufferMut::<T>::with_capacity(values.len());
238-
values
239-
.try_map_masked_into(
240-
mask.bit_buffer(),
241-
&mut buffer.spare_capacity_mut()[..values.len()],
242-
|value| cast_plan.cast(value),
243-
)
244-
.map_err(|idx| {
245-
decimal_cast_error::<F, T>(values[idx], from_decimal_dtype, to_decimal_dtype)
246-
})?;
247-
// SAFETY: try_map_masked_into initializes every lane before returning Ok.
248-
unsafe { buffer.set_len(values.len()) };
249-
buffer.freeze()
251+
values.try_map_masked_into(
252+
mask.bit_buffer(),
253+
&mut buffer.spare_capacity_mut()[..values.len()],
254+
&mut cast,
255+
)?;
250256
}
251-
};
252-
253-
Ok(DecimalArray::new(buffer, to_decimal_dtype, validity).into_array())
257+
}
258+
// SAFETY: the selected map kernel initialized every lane before returning Ok.
259+
unsafe { buffer.set_len(values.len()) };
260+
Ok(buffer.freeze())
254261
}
255262

256263
#[cold]

0 commit comments

Comments
 (0)