Repository navigation
Commit 0c4279a
committed
perf: add a GroupsAccumulator for PivotFirst
PivotFirst only had a row Accumulator, so grouped aggregation went through
DataFusion's GroupsAccumulatorAdapter, which visits every group's state on each
batch, and the accumulator built a ScalarValue for every input row.
The grouped accumulator keeps every group's slots in one flat, group-major
buffer with a validity bitmap, and returns them as the list without copying.
Pivot values are looked up in a typed index built once per aggregate, with
Spark's float rules in a new float_semantics::ieee_equality_key helper. The
ungrouped accumulator shares the index. A pivot list with a repeated value is
rejected at construction, since the serde falls back for it.1 parent 1c6d38f commit 0c4279a
3 files changed
Lines changed: 927 additions & 212 deletions
File tree
- native
- core/src/execution
- spark-expr/src
- agg_funcs
- float_semantics
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3567 | 3567 | | |
3568 | 3568 | | |
3569 | 3569 | | |
3570 | | - | |
3571 | | - | |
3572 | | - | |
| 3570 | + | |
| 3571 | + | |
3573 | 3572 | | |
3574 | 3573 | | |
3575 | 3574 | | |
| |||
3580 | 3579 | | |
3581 | 3580 | | |
3582 | 3581 | | |
3583 | | - | |
3584 | | - | |
| 3582 | + | |
| 3583 | + | |
| 3584 | + | |
| 3585 | + | |
3585 | 3586 | | |
3586 | 3587 | | |
3587 | 3588 | | |
| |||
0 commit comments