Repository navigation
Move scalar functions out of vortex-array into vortex-functions - #10762
robert3005 wants to merge 1 commit into
Conversation
714ed9d to
fd2bbf1
Compare
Merging this PR will regress 6 benchmarks2 warnings
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
0713434 to
e33c436
Compare
595165d to
7644dd2
Compare
dcaa482 to
9383bb2
Compare
vortex-array keeps the scalar function machinery (the `ScalarFnVTable` trait,
binding, reduction, execution, the session registry and kernel registry) and
the few functions its own arrays, validity and expressions are built on:
literal, get_item, pack and select, is_null, is_not_null, not and the binary
operators, and cast, mask and fill_null, which canonical arrays apply to their
own children. That module is renamed from `scalar_fn::fns` to
`scalar_fn::builtin`, matching `ArrayBuiltins`. Select stays because get_item
reads through a selection. Every other scalar function moves to the new
`vortex-functions` crate:
- byte_length, case_when, ext_storage, list_length, list_sum, list_transform,
merge, replace_time_zone, dynamic, dynamic_expression, stat, between, like,
list_contains, variant_get, is_nan and zip, with their expression
constructors in `vortex_functions::expr` (and `expr::bound`)
- the kernels and parent-reduce rules for vortex-array's encodings, including
`Widen`'s between and zip kernels, which `vortex_functions::initialize`
registers with a session's kernel registry
- the `ArrayBuiltins`/`ExprBuiltins` methods for the moved functions, which
become the `ArrayFunctions`/`ExprFunctions` extension traits
`ArrayHaystack`, `LazyBitBufferBuilder`,
`DeduplicatedBuffers::{extend_from_iter, finish}`, `ListScalar::element_values`,
`is_not_null_node`, `build_row_comparator`, `merge_typed_scalar_as_variant` and
the `Widen` compute helpers become public for the moved kernels, and
`ForeignExtDType` is public under `_test-harness` for the moved tests.
The umbrella crate re-exports the moved constructors under `vortex::expr` and
the crate as `vortex::functions`. vortex-array's integration tests and benches
use vortex-functions as a dev-dependency.
The CodSpeed shard plan picks up the benchmarks that move to `vortex-functions`;
their recorded counts and durations move with them.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ps4mp7KF5c4p66aoSoXx5j
Signed-off-by: Claude <[email protected]>
9383bb2 to
29fe7c3
Compare
vortex-array keeps the scalar function machinery (the
ScalarFnVTabletrait, binding, reduction, execution, the session registry and kernel registry) and the few functions its own arrays, validity and expressions are built on: literal, get_item, pack and select, is_null, is_not_null, not and the binary operators, and cast, mask and fill_null, which canonical arrays apply to their own children. That module is renamed fromscalar_fn::fnstoscalar_fn::builtin, matchingArrayBuiltins. Every other scalar function moves to the newvortex-functionscrate:vortex_functions::expr(andexpr::bound)Widen's, whichvortex_functions::initializeregisters with the session's kernel registryArrayBuiltins/ExprBuiltinsmethods for the moved functions, which become theArrayFunctions/ExprFunctionsextension traitsArrayHaystack,LazyBitBufferBuilder,DeduplicatedBuffers::{extend_from_iter, finish},ListScalar::element_values,is_not_null_node,build_row_comparator,merge_typed_scalar_as_variantand theWidencompute helpers become public for the moved kernels;ForeignExtDTypeis public under_test-harnessfor the moved tests.