Repository navigation
perf(aggregate): hold the storage partial directly in ExtensionAggregate - #10612
Conversation
ExtensionAggregate held a nested Accumulator<A> as its partial, so every empty partial (once per group in grouped accumulation), every partial parsed from a scalar, and every merge built or walked a full accumulator: an Arc allocation for the bound aggregate, clones of three dtypes, and a dynamic downcast plus dtype comparison on merge. The partial is now A::Partial, and every vtable call forwards to A with the storage dtype borrowed from the extension dtype, so partial handling allocates nothing. Batches still go through a per-batch Accumulator<A> over the storage array, so kernels and cached statistics of A on the storage encoding keep firing. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01XQcwwu2BajnUtkoph7encr Signed-off-by: Claude <[email protected]>
Merging this PR will regress 32 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | mul_u64_nonnull_avx512 |
29.2 µs | 52.3 µs | -44.22% |
| ❌ | WallTime | deferred_i64_avx512[PerRowPerRow] |
5.7 µs | 10 µs | -43.29% |
| ❌ | WallTime | mul_u64_nonnull_avx2 |
24.4 µs | 39 µs | -37.32% |
| ❌ | WallTime | encode_chunked_neon[i64, 524288] |
23.8 µs | 35.8 µs | -33.35% |
| ❌ | WallTime | infallible_bool_constant_avx2[ConstantPerRow] |
6.9 µs | 10.3 µs | -33.29% |
| ❌ | WallTime | infallible_bool_constant_avx2[PerRowConstant] |
6.9 µs | 10.3 µs | -33.23% |
| ❌ | WallTime | infallible_bool_avx2[i64, PerRowPerRow] |
6.9 µs | 10.3 µs | -33% |
| ❌ | WallTime | deferred_bool_constant_avx2[ConstantPerRow] |
7.2 µs | 10.7 µs | -32.9% |
| ❌ | WallTime | deferred_bool_constant_avx2[PerRowConstant] |
7.4 µs | 10.7 µs | -30.9% |
| ❌ | WallTime | deferred_bool_avx2[i64, PerRowPerRow] |
7.4 µs | 10.7 µs | -30.78% |
| ❌ | Simulation | decompress[u8, (4000, 256)] |
31.6 µs | 44.7 µs | -29.29% |
| ❌ | WallTime | mul_u64_nonnull_neon |
21.3 µs | 28.4 µs | -24.96% |
| ❌ | WallTime | deferred_i64_avx2[PerRowPerRow] |
7.7 µs | 10 µs | -23.2% |
| ❌ | WallTime | compare_float_neon |
6.4 µs | 8.1 µs | -21.72% |
| ❌ | Simulation | patched_take_10_stratified |
56.9 µs | 72.1 µs | -21.1% |
| ❌ | WallTime | deferred_i64_neon[PerRowPerRow] |
12.1 µs | 15 µs | -19.72% |
| ❌ | WallTime | take_u16_from_65536_neon[1048576] |
745.7 µs | 898.3 µs | -16.99% |
| ❌ | WallTime | mul_i16_nonnull_avx512 |
13.6 µs | 16.3 µs | -16.76% |
| ❌ | WallTime | take_u16_from_65536_neon[65536] |
47 µs | 56.4 µs | -16.63% |
| ❌ | Simulation | take_map[(0.1, 1.0)] |
222.5 µs | 263.8 µs | -15.68% |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing rob/extension-aggregate-overhead (1ea5c90) with ji/stoic-pascal-p9i2a9 (3db29a5)
Footnotes
-
161 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
3ce4231
into
ji/stoic-pascal-p9i2a9
ExtensionAggregate held a nested Accumulator as its partial, so every empty partial (once per group in grouped accumulation), every partial parsed from a scalar, and every merge built or walked a full accumulator: an Arc allocation for the bound aggregate, clones of three dtypes, and a dynamic downcast plus dtype comparison on merge.
The partial is now A::Partial, and every vtable call forwards to A with the storage dtype borrowed from the extension dtype, so partial handling allocates nothing. Batches still go through a per-batch Accumulator over the storage array, so kernels and cached statistics of A on the storage encoding keep firing.