Skip to content

Commit 0291778

Browse files
authored
Attach RowFn validity directly to canonical output (#10742)
## Summary - Tracking Issue: #9128 - Supersedes #10674 - Related discussion: #9694 - Independent float32 candidate: #10741 Attaches input validity directly to canonical RowFn output using the existing helper. Preserves kernel-output validation, relabeling, outer nullability, and the non-canonical mask fallback. `values.mask(valid)` builds and optimizes a `ScalarFnArray` to dispatch through parent-reduce rules. The benchmark-driven bypass stays inside RowFn to avoid adding canonical-specific dispatch to the generic builtin. It can be removed if `.mask()` gains equivalent direct dispatch. This revisits the abstraction concern in #9694. **Draft pending fresh isolated benchmarks on this head.** Saved results include regressions and do not establish a general speedup. <details> <summary>Saved benchmark evidence and validation</summary> The isolated validity snapshot `b1961452ea3505d4e2588e34b14c20ea177ed175` was compared with frozen `develop` at `a1693389301e446eec41373cb274d98a303d3ca0`. Measurements used Rust 1.99.1 / LLVM 23.1.1, 16 CGUs, no LTO, CPU 2, and 21 balanced paired blocks on Graviton3 C7g and Sapphire Rapids C7i. This was native wall time, not CodSpeed simulation. For nullable u64 addition with 64 rows and a right-hand constant, median paired time ratios were: | Target | Candidate / baseline time | 95% paired bootstrap interval | | --- | ---: | ---: | | NEON | 0.798 | 0.796–0.802 | | AVX2 | 0.850 | 0.846–0.852 | | AVX-512 | 0.846 | 0.844–0.855 | The isolated snapshot also had flagged regressions outside that case: roughly 3.4% slower ARM u64 multiply and up to 4.2% slower AVX2 byte comparisons. No regressions crossed the 3% flag on AVX-512 in that run. These movements remain part of the review, even where unchanged code is sensitive to compiler context or binary layout. A separate combined packing/validity candidate at `8d8ef8151454bbc73e1cd0b236567542a1401cbe`, based on `d834a11faa79370b4d2e0bbcc73195f451a8a212`, retained the small-batch gain but regressed nullable u64 add by about 12% at 8,192 rows and 18% at 32,768 rows on AVX2. Profiling found a byte-identical retry hot loop with essentially unchanged instructions per operation and more cycles. The precise placement-sensitive mechanism remains unresolved. That combined result does not isolate this PR. The combined candidate passed 225 RowFn tests and 232 binary-function tests on that newer base, including output validation, allocator, and deferred-error coverage. Separate native probes checked null-row arithmetic errors and comparison edge cases. The changed file passed the pinned formatter. These are saved checks, not fresh validation of this individual PR. This PR is rebased onto `c750fbd80882f78efd6a2703bc0f3e17f1aca9f9`. The patch is unchanged by the rebase. Tests and benchmarks were not rerun after it. </details> --------- Signed-off-by: Connor Tsui <[email protected]>
1 parent dcb4f50 commit 0291778

1 file changed

Lines changed: 21 additions & 1 deletion

File tree

  • vortex-array/src/scalar_fn/unstable/row/batch/execute

‎vortex-array/src/scalar_fn/unstable/row/batch/execute/dense.rs‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ use vortex_mask::MaskValuesRef;
99
use super::super::RowFnExecutionArgs;
1010
use super::super::args::BorrowedRowFnArgs;
1111
use crate::ArrayRef;
12+
use crate::Canonical;
1213
use crate::ExecutionCtx;
14+
use crate::IntoArray;
15+
use crate::arrays::masked::mask_validity_canonical;
1316
use crate::builtins::ArrayBuiltins;
17+
use crate::canonical::AnyCanonical;
1418
use crate::scalar_fn::unstable::row::execute::DenseAttempt;
1519
use crate::validity::Validity;
1620

@@ -107,7 +111,23 @@ impl RowFnExecutionArgs {
107111
Validity::NonNullable | Validity::AllValid => {
108112
self.finalize_output(values, self.row_count)
109113
}
110-
Validity::Array(valid) => self.finalize_output(values.mask(valid)?, self.row_count),
114+
Validity::Array(valid) => {
115+
// `values.mask(valid)` builds and optimizes a ScalarFnArray to dispatch through
116+
// parent-reduce rules. Benchmarks motivated bypassing that setup here, keeping the
117+
// exception internal instead of adding canonical-specific dispatch to the generic
118+
// builtin. Remove this bypass if `.mask()` gains equivalent direct dispatch.
119+
// https://github.com/vortex-data/vortex/pull/10742
120+
let values = match values.as_opt::<AnyCanonical>() {
121+
Some(canonical) => mask_validity_canonical(
122+
Canonical::from(canonical),
123+
Validity::Array(valid),
124+
ctx,
125+
)?
126+
.into_array(),
127+
None => values.mask(valid)?,
128+
};
129+
self.finalize_output(values, self.row_count)
130+
}
111131
Validity::AllInvalid => {
112132
unreachable!("all-invalid validity is handled before dense row execution")
113133
}

0 commit comments

Comments
 (0)