Skip to content

Commit d81bd4c

Browse files
authored
Compare primitive values through the shared RowFn executor (#10676)
Tracking Issue: #9128 ## Summary Moves primitive comparisons onto the shared RowFn executor from #10675, so comparison code only selects the predicate while the framework handles decoding, constants, validity, and Boolean output. ## Changes Preserves the public `Binary` identity and floating-point total ordering. Uses borrowed operands and dtypes, and moves a constant left operand to the right with the swapped operator. Relies on the existing `primitive_comparisons_across_bitmap_words` test for all six swapped operators and keeps the existing comparison benchmarks. Based on #10675. Keeps the wide-type, sliced-input, bitmap-boundary, NaN, and signed-zero tests added upstream in #10843. ## Performance CodSpeed compares this PR with #10675 (`b0a64dd`). Nine primitive comparison benchmarks are 12% to 31% faster in walltime, led by `compare_u64_avx2` and `compare_float_avx2` at 31% and 28%, and no comparison benchmark regresses. These gains depend on #10675. #11220 measured the same two commits directly on `develop`. Without #10675, comparisons on small arrays were 10% to 16% slower than `develop` in CPU simulation, and only two walltime gains remained, at 16% and 18%. The #10675 description has the full comparison. The baseline already includes the private NEON packing kernel from #10843, which this PR replaces with RowFn execution. So the gains above are measured against that improved baseline, and the old packing gains aren't credited to this PR. <details> <summary>CodSpeed comparison benchmarks against #10675</summary> CodSpeed marks the walltime runners as unreliable, but all nine cases move in the same direction. | Benchmark | Faster by | | --- | ---: | | `compare_u64_avx2` | 30.8% | | `compare_float_avx2` | 28.2% | | `lt_i64_nullable_avx2` | 18.4% | | `compare_int_constant_neon` | 16.2% | | `compare_int_eq_avx2` | 15.4% | | `compare_f32_avx2` | 15.4% | | `compare_int_constant_avx2` | 15.3% | | `compare_int_avx2` | 13.5% | | `compare_int_nullable_avx2` | 12.5% | </details> <details> <summary>Historical EC2 accumulated-stack measurements</summary> These runs measured an earlier accumulated stack, not this layer in isolation or the current head. They showed gains on some configurations and regressions on others. Ratios are the median of 21 paired candidate/baseline elapsed-time ratios. Below 1 is faster. This compares the earlier complete migration, M0 `ddf8a16df6b7d7c20cfd860e5f23438d97ca4238`, with D `0291778db36809f77ce0219bf9850506d3cba3b6`. That baseline predates #10843. The array-pair cases use all-valid nullable inputs. The addition case is an arithmetic control whose algorithm was unchanged. | Workload | C7g NEON | C6a AVX2 | C7i AVX2 build | C7i AVX-512 build | | --- | ---: | ---: | ---: | ---: | | `f32` array-pair Gte, 8,192 rows | 1.014 | 1.040 | 0.873 | 1.036 | | `i64` array-pair Gte, 8,192 rows | 0.900 | 1.139 | 0.929 | 0.943 | | Nullable `i64` add, 32,768 rows | 1.002 | 1.123 | 1.154 | 1.004 | For example, the old stack took about 10% less time for the ARM `i64` comparison, but about 14% more for the same case on AVX2-only C6a. C6a nullable `i64` addition took about 12% more time. These whole-executable measurements do not establish that the comparison algorithm caused movement in arithmetic controls. In the four-label matrix, D ran before M0 in 11 blocks and after it in ten. The experimental one-pass decoder variants are excluded from this table. EC2 C7g (Neoverse V1), C6a (EPYC 7R13), and C7i (Xeon Platinum 8488C). Rust 1.99.1 / LLVM 23.1.1, bench profile with 16 CGUs and no LTO, MiMalloc, CPU 2 pinned. The same 455-case public-path microbenchmark suite ran serially across hosts, with 21 paired blocks, 10 ms per case, A/A calibration, and a separate control executable. These are microbenchmarks, not query results. C7i can dispatch to AVX-512 at runtime even in its AVX2 build. Saved evidence is in the local investigation archive `row-fn-evaluation-20261026`: `manifest.json`, `*-matrix-*-medians.csv`, `*-matrix-*-summary.json`, and `results/<host>/results/matrix-*.json`. </details> <details> <summary>Confirmed independent packing gains, now in develop via #10843</summary> The ARM-only private-kernel change reduced long array-pair cohort time on Graviton3 as follows. Each cohort contains 32 cases, including partially valid inputs and lengths around 16K. | Cohort | Confirmation elapsed-time ratio | Paired 95% interval | Time reduction | | --- | ---: | --- | ---: | | `i64` | 0.846 | 0.843 to 0.856 | 15.4% | | `u64` | 0.860 | 0.858 to 0.864 | 14.0% | | `f64` | 0.882 | 0.880 to 0.886 | 11.8% | Baseline `a59294e35e7109fac894a7a38a68f42db0c3c967`, candidate `f018aaeef739145e1bc50079a441c5d9fd53404a`. The cohort uses a geometric mean of case ratios within each block, then the median across blocks. Two 21-block sessions used complementary AB/BA orders, the same full 455-case suite, and 1,000 single-iteration samples per case. The initial fixed-order source comparisons were discarded. The AVX2 builds had identical `.text` bytes, so no x86 speedup was attributed to this change. See #10843 for the harness and full methodology. These gains required neither this PR nor #10675. </details> On `e724610`, CI passed all-target, all-feature Clippy with warnings denied and the Linux, CPU feature, sanitizer, and Miri test jobs. --------- Signed-off-by: Connor Tsui <[email protected]>
1 parent df153c0 commit d81bd4c

33 files changed

Lines changed: 742 additions & 372 deletions

‎vortex-array/benches/compare.rs‎

Lines changed: 11 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -3,22 +3,18 @@
33

44
//! Benchmarks for the binary comparison path, over every array kind it accepts.
55
//!
6-
//! The primitive cases carry `#[cpu_features]`, so they are measured on every walltime
7-
//! CPU-feature leg rather than in simulation. Each is written once and compiled differently
8-
//! per leg: today the primitive comparison path is a portable lane kernel, and how well it
9-
//! auto-vectorizes is decided by the build. That is the baseline a hand-written kernel
10-
//! selected through `cfg(target_feature)` has to beat, measured on the silicon it would run
11-
//! on.
6+
//! Primitive cases use `#[cpu_features]` so walltime runs exercise the multiversioned Boolean
7+
//! collector used by [`RowFn`] on each CPU-feature runner. Simulation does not measure those
8+
//! target-specific paths.
129
//!
13-
//! Every case here compares one array against another, which is the shape the path is tuned for.
14-
//! The three constant cases that remain — boolean, integer, and string — are a regression guard on
15-
//! constant handling rather than coverage of it: a constant operand is decoded the same way
16-
//! whatever it holds. Constant orientation is measured once, in `binary_ops.rs`.
10+
//! Array-pair cases cover the comparison kernels. Boolean, integer, and string constant cases are
11+
//! regression guards rather than exhaustive constant coverage. Shared constant setup is measured
12+
//! in `binary_ops.rs` with both operand orientations, so it does not need a full matrix here.
1713
//!
18-
//! The boolean, decimal, string, and struct cases are not tagged. A wider vector register is
19-
//! not what decides them: booleans are already word-at-a-time over a bitmap, decimals are
20-
//! `i128`, and the string and struct cases are dominated by view chasing and per-field
21-
//! dispatch. They stay in simulation.
14+
//! Boolean, decimal, string, and struct cases stay in simulation. They use bitmap operations,
15+
//! `i128` comparisons, view access, or per-field dispatch rather than the multiversioned collector.
16+
//!
17+
//! [`RowFn`]: vortex_array::scalar_fn::unstable::row::RowFn
2218
2319
#![expect(clippy::unwrap_used)]
2420

@@ -66,7 +62,7 @@ fn main() {
6662
divan::main();
6763
}
6864

69-
// Sized to keep CodSpeed simulation under 1ms per benchmark.
65+
// Keep simulated workloads within CodSpeed's per-case runtime limit.
7066
const ARRAY_SIZE: usize = 8_192;
7167

7268
fn bench_compare(bencher: Bencher, lhs: ArrayRef, rhs: ArrayRef, op: Operator) {

‎vortex-array/benches/row_fn_output.rs‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -55,11 +55,9 @@ const INPUT_SHAPES: &[InputShape] = &[InputShape::PerRowPerRow];
5555

5656
/// Constant operands are measured on `i64` alone, by the `*_constant` benchmarks below.
5757
///
58-
/// The row executor decodes and orients a constant operand the same way whatever the row kernel
59-
/// does with the value, so running both constant orientations under every element type and every
60-
/// output mode measures one path repeatedly, three walltime legs at a time. One infallible pair and
61-
/// one deferred pair keep it covered: those two differ in how a constant row reaches the kernel,
62-
/// and that is the difference worth watching.
58+
/// Constant handling is shared across row kernels, so repeating both orientations for every
59+
/// element type and output mode would duplicate coverage. Infallible and deferred cases remain
60+
/// separate because they pass constants through different execution paths.
6361
const CONSTANT_SHAPES: &[InputShape] = &[InputShape::PerRowConstant, InputShape::ConstantPerRow];
6462

6563
#[derive(Clone, Copy, Debug)]

‎vortex-array/benches/row_fn_utf8_input.rs‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ struct Case {
7474

7575
impl Display for Case {
7676
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
77-
// Preserve the report names from the former const-length and layout parameters.
77+
// Keep report names stable so benchmark history follows the same input layout and length.
7878
let layout = match self.layout {
7979
Layout::VarBin => "VarBin",
8080
Layout::VarBinView => "VarBinView",
@@ -89,7 +89,9 @@ fn cases() -> Vec<Case> {
8989
.iter()
9090
.flat_map(|&len| LAYOUTS.iter().map(move |&layout| Case { len, layout }))
9191
.filter(|case| {
92-
// VarBin varies on unchanged code, except for short values on the NEON leg.
92+
// Run the full layout matrix locally. CI retains only VarBin cases whose walltime was
93+
// stable enough to distinguish code changes from noise. Recheck stability before
94+
// expanding the CI matrix.
9395
env!("VORTEX_BENCH_VARIANT") == "local"
9496
|| matches!(case.layout, Layout::VarBinView)
9597
|| (env!("VORTEX_BENCH_VARIANT") == "neon" && case.len == 8)

‎vortex-array/src/scalar_fn/fns/binary/compare/mod.rs‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,16 +4,19 @@
44
//! Native comparison kernels.
55
//!
66
//! [`execute_compare`] dispatches on the logical [`DType`] of its operands and evaluates every
7-
//! comparison directly over Vortex canonical arrays: bit buffers for booleans, lane kernels from
8-
//! `vortex-compute` for primitives and decimals, binary views for strings/bytes, and a row-wise
9-
//! comparator for nested types. There is no Arrow fallback.
7+
//! comparison directly over Vortex canonical arrays: bit buffers for booleans, [`RowFn`] for
8+
//! primitives, lane kernels from `vortex-compute` for decimals, binary views for strings/bytes, and
9+
//! a row-wise comparator for nested types. There is no Arrow fallback.
1010
//!
11-
//! Floating point values compare with Vortex's total ordering (`NaN` is the largest value,
12-
//! `-0.0 < +0.0`, and equality is bitwise), matching [`Scalar`] comparison semantics.
11+
//! Floating-point values use the same total ordering as [`Scalar`]: negative NaNs sort below
12+
//! negative infinity, positive NaNs sort above positive infinity, and `-0.0 < +0.0`. Equality is
13+
//! bitwise, so signed zeros and distinct NaN payloads remain distinct.
1314
//!
1415
//! Binary scalar functions can execute through encoding-specific kernels before reaching the native
1516
//! dispatcher. [`CompareExecuteAdaptor`] connects these execution paths to [`CompareKernel`], which
1617
//! lets an encoding compare values without first converting them to canonical arrays.
18+
//!
19+
//! [`RowFn`]: crate::scalar_fn::unstable::row::RowFn
1720
1821
use std::cmp::Ordering;
1922

@@ -209,7 +212,7 @@ fn compare_arrays(
209212
)
210213
.into_array()),
211214
DType::Bool(_) => boolean::compare_bool(lhs, rhs, op, nullability, ctx),
212-
DType::Primitive(..) => primitive::compare_primitive(lhs, rhs, op, nullability, ctx),
215+
DType::Primitive(..) => primitive::compare_primitive(lhs, rhs, op, ctx),
213216
DType::Decimal(..) => decimal::compare_decimal(lhs, rhs, op, nullability, ctx),
214217
DType::Utf8(_) | DType::Binary(_) => bytes::compare_bytes(lhs, rhs, op, nullability, ctx),
215218
DType::Struct(..) | DType::List(..) | DType::FixedSizeList(..) | DType::Map(..) => {

0 commit comments

Comments
 (0)