Skip to content

Commit f04a28c

Browse files
authored
Avoid a per-run zip_eq call in run-end bool decoding (#10962)
## Summary The prefill loops in the run-end bool decoder iterate with `itertools::zip_eq`. `ZipEq::next` is generic and has no `#[inline]`, so rustc emits each instantiation in a single codegen unit, and a loop in a different unit has to call it out of line once per run. #10676 doesn't touch the decoder, but it moves the decoder to another codegen unit in the bench profile (16 codegen units, no LTO), and `decode_bool_nullable` slows down by 17 to 25% in the CodSpeed simulation. ## Changes The two prefill loops now use `zip`, like the sequential paths in the same file. `runend_decode_bools` asserts once that the run ends and the values have the same length. This keeps the panic that `zip_eq` raised on a mismatch, and the few-runs paths now get it too. Every caller passes the children of a `RunEndArray`, which already checks the lengths at construction. The x86-64 assembly of `vortex-array`, built with the CodSpeed simulation flags at the head of #10676 plus this change, has no call to `ZipEq::next`, even though the decoder is still in the other codegen unit. On `develop`, the decoder's CodSpeed numbers already match the inlined case, so this PR on its own may not show a change. Signed-off-by: Connor Tsui <[email protected]>
1 parent 891c01a commit f04a28c

1 file changed

Lines changed: 16 additions & 3 deletions

File tree

‎vortex-array/src/arrays/runend/decompress_bool.rs‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@
66
//! Uses an adaptive strategy that pre-fills the buffer with the majority value
77
//! (0s or 1s) and only fills the minority runs, minimizing work for skewed distributions.
88
9-
use itertools::Itertools;
109
use vortex_buffer::BitBuffer;
1110
use vortex_buffer::BitBufferMut;
1211
use vortex_error::VortexResult;
@@ -31,13 +30,27 @@ use crate::validity::Validity;
3130
const PREFILL_RUN_THRESHOLD: usize = 32;
3231

3332
/// Decodes run-end encoded boolean values into a flat `BoolArray`.
33+
///
34+
/// # Panics
35+
///
36+
/// Panics if `ends` and `values` have different lengths.
3437
pub fn runend_decode_bools(
3538
ends: PrimitiveArray,
3639
values: BoolArray,
3740
offset: usize,
3841
length: usize,
3942
ctx: &mut ExecutionCtx,
4043
) -> VortexResult<ArrayRef> {
44+
// NB: The decode loops pair run ends with values through `zip`, which stops at the shorter
45+
// input, so this check must stay. Do not switch the loops to `itertools::zip_eq`: `ZipEq::next`
46+
// has no `#[inline]`, so rustc can place it in another codegen unit, and the loop then calls it
47+
// out of line once per run.
48+
assert_eq!(
49+
ends.len(),
50+
values.len(),
51+
"run ends and values must have the same length"
52+
);
53+
4154
let validity = values
4255
.as_ref()
4356
.validity()?
@@ -171,7 +184,7 @@ fn decode_bool_non_nullable(
171184
let mut decoded = BitBufferMut::full(prefill, length);
172185
let mut current_pos = 0usize;
173186

174-
for (end, value) in run_ends.zip_eq(values.iter()) {
187+
for (end, value) in run_ends.zip(values.iter()) {
175188
if end > current_pos && value != prefill {
176189
// SAFETY: current_pos < end <= length == decoded.len()
177190
unsafe { decoded.fill_range_unchecked(current_pos, end, value) };
@@ -203,7 +216,7 @@ fn decode_bool_nullable(
203216
let mut decoded_validity = BitBufferMut::full(prefill_valid, length);
204217
let mut current_pos = 0usize;
205218

206-
for (end, (value, is_valid)) in run_ends.zip_eq(values.iter().zip(validity_mask.iter())) {
219+
for (end, (value, is_valid)) in run_ends.zip(values.iter().zip(validity_mask.iter())) {
207220
if end > current_pos {
208221
// SAFETY: current_pos < end <= length == decoded.len() == decoded_validity.len()
209222
if is_valid != prefill_valid {

0 commit comments

Comments
 (0)