Repository navigation
Commit 6a4e997
committed
Fix aggregate accumulator capacity accounting (apache#24099)
## Which issue does this PR close?
* Part of apache#23393
## Rationale for this change
Two aggregate accumulators underreport owned memory in their `size()`
implementations:
* `CountGroupsAccumulator` calculates vector backing storage using the
wrong element type.
* `SlidingDistinctSumAccumulator` reports only the accumulator struct
size and omits its owned hash map allocation.
This change corrects those capacity-based estimates while preserving the
different `Accumulator::size()` and `GroupsAccumulator::size()`
self-inclusion contracts.
## What changes are included in this PR?
* Add `vec_capacity_bytes` to calculate a vector’s backing allocation
using its actual element type.
* Update `CountGroupsAccumulator::size()` to report the capacity-based
allocation of its counts vector while remaining self-excluded.
* Update `SlidingDistinctSumAccumulator::size()` to include
`size_of_val(self)` and an estimate of its hash map’s allocated entry
capacity.
* Document that the hash map estimate excludes implementation-specific
control bytes.
## Are these changes tested?
Yes. This PR adds the following unit tests:
* `count_groups_size_includes_vec_capacity`
* `vec_capacity_bytes_uses_element_type`
* `sliding_distinct_sum_size_includes_hash_map_capacity`
The tests verify that:
* grouped count size grows when vector backing storage is allocated;
* vector capacity accounting uses the vector’s element type;
* sliding distinct-sum size includes hash map capacity, grows after the
map expands, and continues to reflect retained capacity after entries
are retracted.
## Are there any user-facing changes?
No user-facing API or aggregate result changes are intended. This PR
only corrects internal memory-size estimates used for aggregate
accounting.
## LLM-generated code disclosure
This PR includes LLM-generated code and comments. All LLM-generated
content has been manually reviewed.1 parent e05aa1f commit 6a4e997
2 files changed
Lines changed: 60 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| 32 | + | |
32 | 33 | | |
33 | 34 | | |
34 | 35 | | |
| |||
774 | 775 | | |
775 | 776 | | |
776 | 777 | | |
777 | | - | |
| 778 | + | |
778 | 779 | | |
779 | 780 | | |
780 | 781 | | |
| |||
932 | 933 | | |
933 | 934 | | |
934 | 935 | | |
| 936 | + | |
| 937 | + | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
935 | 953 | | |
936 | 954 | | |
937 | 955 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
51 | | - | |
| 51 | + | |
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| |||
671 | 671 | | |
672 | 672 | | |
673 | 673 | | |
674 | | - | |
| 674 | + | |
| 675 | + | |
675 | 676 | | |
676 | 677 | | |
677 | 678 | | |
| |||
722 | 723 | | |
723 | 724 | | |
724 | 725 | | |
725 | | - | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
726 | 730 | | |
727 | 731 | | |
728 | 732 | | |
| |||
750 | 754 | | |
751 | 755 | | |
752 | 756 | | |
| 757 | + | |
| 758 | + | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
| 766 | + | |
| 767 | + | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
| 778 | + | |
| 779 | + | |
| 780 | + | |
| 781 | + | |
| 782 | + | |
| 783 | + | |
| 784 | + | |
| 785 | + | |
| 786 | + | |
| 787 | + | |
| 788 | + | |
| 789 | + | |
| 790 | + | |
753 | 791 | | |
754 | 792 | | |
755 | 793 | | |
| |||
0 commit comments