Repository navigation
Commit 29fe7c3
committed
Move scalar functions out of vortex-array into vortex-functions
vortex-array keeps the scalar function machinery (the `ScalarFnVTable` trait,
binding, reduction, execution, the session registry and kernel registry) and
the few functions its own arrays, validity and expressions are built on:
literal, get_item, pack and select, is_null, is_not_null, not and the binary
operators, and cast, mask and fill_null, which canonical arrays apply to their
own children. That module is renamed from `scalar_fn::fns` to
`scalar_fn::builtin`, matching `ArrayBuiltins`. Select stays because get_item
reads through a selection. Every other scalar function moves to the new
`vortex-functions` crate:
- byte_length, case_when, ext_storage, list_length, list_sum, list_transform,
merge, replace_time_zone, dynamic, dynamic_expression, stat, between, like,
list_contains, variant_get, is_nan and zip, with their expression
constructors in `vortex_functions::expr` (and `expr::bound`)
- the kernels and parent-reduce rules for vortex-array's encodings, including
`Widen`'s between and zip kernels, which `vortex_functions::initialize`
registers with a session's kernel registry
- the `ArrayBuiltins`/`ExprBuiltins` methods for the moved functions, which
become the `ArrayFunctions`/`ExprFunctions` extension traits
`ArrayHaystack`, `LazyBitBufferBuilder`,
`DeduplicatedBuffers::{extend_from_iter, finish}`, `ListScalar::element_values`,
`is_not_null_node`, `build_row_comparator`, `merge_typed_scalar_as_variant` and
the `Widen` compute helpers become public for the moved kernels, and
`ForeignExtDType` is public under `_test-harness` for the moved tests.
The umbrella crate re-exports the moved constructors under `vortex::expr` and
the crate as `vortex::functions`. vortex-array's integration tests and benches
use vortex-functions as a dev-dependency.
The CodSpeed shard plan picks up the benchmarks that move to `vortex-functions`;
their recorded counts and durations move with them.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ps4mp7KF5c4p66aoSoXx5j
Signed-off-by: Claude <[email protected]>1 parent 26a12a6 commit 29fe7c3
430 files changed
Lines changed: 5027 additions & 4413 deletions
File tree
- encodings
- alp
- src
- alp_rd
- compute
- alp
- compute
- bytebool/src
- datetime-parts
- src/compute
- decimal-byte-parts
- benches
- src/decimal_byte_parts
- compute
- fastlanes
- benches
- src
- bitpacking
- array
- compute
- vtable
- delta
- compute
- vtable
- for_
- compute
- vtable
- rle
- compute
- probe
- vtable
- tests
- fsst
- benches
- src
- compute
- dfa
- onpair
- src
- compute
- tests
- parquet-variant
- src
- pco/src
- compute
- sequence
- src
- compute
- sparse
- benches
- src
- compute
- zigzag/src
- compute
- zstd/src
- compute
- fuzz
- src
- array
- scripts
- vortex-array
- benches
- src
- aggregate_fn/fns
- all_non_distinct
- is_constant
- mean
- sum_v2
- arrays
- bool
- compute
- vtable
- chunked
- compute
- constant/compute
- decimal
- compute
- vtable
- dict
- compute
- vtable
- extension
- compute
- vtable
- filter
- fixed_size_list
- compute
- vtable
- listview
- compute
- tests
- vtable
- list
- compute
- map
- compute
- vtable
- masked/compute
- null/compute
- patched
- compute
- vtable
- primitive
- compute
- vtable
- runend
- compute
- probe
- scalar_fn
- vtable
- slice
- struct_
- compute
- vtable
- union/compute
- varbinview
- compute
- vtable
- varbin
- compute
- vtable
- variant
- vtable
- widen
- compute
- tests
- array
- builders
- dict
- compute/conformance
- dtype/extension
- expr
- traversal
- extension/datetime
- optimizer
- scalar_fn
- builtin
- binary
- compare
- numeric
- cast
- fill_null
- mask
- not
- fns
- internal
- unstable/row/batch
- scalar
- typed_view
- primitive
- stats
- test_harness
- execution_paths
- trace
- tests
- vortex-arrow/src/executor
- vortex-cuda/src/dynamic_dispatch
- vortex-datafusion/src
- convert/exprs
- v2
- vortex-duckdb/src
- convert
- vortex-expr
- src
- analysis
- forms
- stats
- rewrite
- transform
- vortex-ffi/src
- vortex-file
- src
- v2
- vortex-functions
- benches
- src
- between
- like
- list_contains
- prepared
- list_transform
- variant_get
- zip
- vortex-jni/src
- vortex-json
- src
- vortex-layout
- examples/reverse_index
- src
- layouts
- dict
- indexed
- list
- struct_
- zoned
- aggregates/bloom_filter
- plan/physical
- plans
- scan
- v2
- vortex-python/src
- arrays
- expr
- vortex-spatial
- src/prune
- vortex-tensor/src
- vortex
- src
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
16 | 17 | | |
17 | 18 | | |
18 | 19 | | |
| |||
322 | 323 | | |
323 | 324 | | |
324 | 325 | | |
| 326 | + | |
325 | 327 | | |
326 | 328 | | |
327 | 329 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| 27 | + | |
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
13 | 12 | | |
14 | 13 | | |
15 | 14 | | |
16 | 15 | | |
17 | 16 | | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | 17 | | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| |||
199 | 199 | | |
200 | 200 | | |
201 | 201 | | |
202 | | - | |
203 | 202 | | |
204 | 203 | | |
205 | | - | |
206 | | - | |
207 | 204 | | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
208 | 208 | | |
209 | 209 | | |
210 | 210 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
| 9 | + | |
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
15 | | - | |
16 | | - | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
144 | 144 | | |
145 | 145 | | |
146 | 146 | | |
147 | | - | |
148 | | - | |
149 | | - | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
150 | 150 | | |
151 | 151 | | |
152 | 152 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | | - | |
| 9 | + | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
61 | | - | |
| 61 | + | |
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
14 | | - | |
15 | | - | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
| 9 | + | |
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
8 | | - | |
| 7 | + | |
| 8 | + | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| |||
0 commit comments