Repository navigation
Commit 5941552
authored
perf: null-free fast path for COUNT(DISTINCT) primitive accumulator (apache#23956)
## Which issue does this PR close?
- Closes apache#23955.
## Rationale for this change
`PrimitiveDistinctCountAccumulator::update_batch` does a per-element
validity check on every row even when the array has no nulls. Its own
`merge_batch` already inserts wholesale from the values buffer, so the
same fast path applies to `update_batch`. Inspired by the null-free fast
paths added to `percentile_cont`/`median` in apache#23954.
## What changes are included in this PR?
- When `null_count() == 0`, insert directly from the values buffer
(`self.values.extend(arr.values().iter().copied())`) instead of the
per-element `Option` check.
- Unit test asserting the fast and general paths agree (a dense array
and a null-containing array with the same non-null values yield the same
distinct count).
## Benchmarks
`count_distinct` benchmark, null-free 8192-row batches:
```
count_distinct i64 80% distinct -67.0% (54.9 -> 18.1 us)
count_distinct i64 99% distinct -66.0% (54.9 -> 18.7 us)
count_distinct u32 80% distinct -66.9% (53.9 -> 17.8 us)
count_distinct u32 99% distinct -65.3% (53.3 -> 18.5 us)
count_distinct i32 80% distinct -66.9% (54.1 -> 17.9 us)
count_distinct i32 99% distinct -66.2% (53.2 -> 18.0 us)
```
The bitmap-backed cases (u8/i8/u16/i16) and the grouped-accumulator
cases don't use this path and are unchanged, as expected.
## Are these changes tested?
Yes — new unit test; existing count-distinct tests pass.
## Are there any user-facing changes?
No.1 parent 043d97f commit 5941552
1 file changed
Lines changed: 48 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| |||
86 | 87 | | |
87 | 88 | | |
88 | 89 | | |
89 | | - | |
90 | | - | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
91 | 96 | | |
92 | | - | |
93 | | - | |
| 97 | + | |
| 98 | + | |
94 | 99 | | |
95 | 100 | | |
96 | 101 | | |
| |||
617 | 622 | | |
618 | 623 | | |
619 | 624 | | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
0 commit comments