Repository navigation
Commit 22e02f1
authored
Push decimal aggregates through Widen (#11004)
Tracking Issue: #10701
Stacked on #11003. #11005 builds on it. No release gate.
## Summary
Decimal widening preserves order, equality and validity, so ordered
aggregates can use the child instead of copying it first. Sum also needs
the logical result precision: using the child's sum dtype could overflow
a result that should fit.
## Changes
Ordered aggregates widen their child partials, and MinMax reuses exact
cached child extrema before scanning. Decimal `Sum` runs in the logical
sum dtype only when the legacy `Stat::Sum` path combines independent
batch totals. `SumV2` and `Sum` without `skip_nans` keep the fallback
because their running sum can cancel a batch total that would overflow
on its own. Tests compare running partials against fresh canonical
batches so cached statistics cannot hide kernel work.
<details>
<summary>Historical local benchmark results</summary>
These retained measurements predate this style pass and rebase. They do
not measure the current head.
`cargo bench -p vortex-array --bench widen_decimal`, Apple M4, medians.
The numbers were measured at the top of this stack before the split, so
they cover the whole stack, not this layer alone. The baseline is the
canonical input, and the candidate is the same values as a `Widen`
(child precision 5 under precision 10, 9 under 18, and 18 under 38).
CodSpeed on this PR measures the layer itself.
| bench | p10 canonical | p10 Widen | p18 canonical | p18 Widen | p38
canonical | p38 Widen |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| `min_max` | 14.93 µs | 15.12 µs | 14.62 µs | 15.12 µs | 21.58 µs |
14.95 µs |
`sum` medians are within 8% of the canonical input at every precision.
</details>
## API Changes
None.
Signed-off-by: Connor Tsui <[email protected]>1 parent c7f9b4c commit 22e02f1
4 files changed
Lines changed: 488 additions & 31 deletions
File tree
- vortex-array/src/arrays/widen
- compute
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
4 | 9 | | |
5 | 10 | | |
6 | 11 | | |
7 | 12 | | |
| 13 | + | |
8 | 14 | | |
| 15 | + | |
9 | 16 | | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
10 | 26 | | |
11 | 27 | | |
12 | 28 | | |
| |||
15 | 31 | | |
16 | 32 | | |
17 | 33 | | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
| 34 | + | |
23 | 35 | | |
24 | 36 | | |
25 | 37 | | |
| |||
34 | 46 | | |
35 | 47 | | |
36 | 48 | | |
37 | | - | |
38 | | - | |
39 | | - | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
40 | 57 | | |
41 | 58 | | |
42 | 59 | | |
43 | | - | |
44 | 60 | | |
45 | 61 | | |
46 | 62 | | |
47 | 63 | | |
48 | | - | |
49 | | - | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
50 | 99 | | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
| 100 | + | |
| 101 | + | |
55 | 102 | | |
56 | | - | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
57 | 116 | | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
58 | 166 | | |
59 | 167 | | |
60 | 168 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
4 | 9 | | |
5 | 10 | | |
6 | 11 | | |
| 12 | + | |
7 | 13 | | |
8 | 14 | | |
9 | 15 | | |
| |||
32 | 38 | | |
33 | 39 | | |
34 | 40 | | |
| 41 | + | |
35 | 42 | | |
36 | 43 | | |
37 | 44 | | |
38 | 45 | | |
39 | 46 | | |
40 | 47 | | |
| 48 | + | |
41 | 49 | | |
42 | 50 | | |
43 | 51 | | |
44 | 52 | | |
45 | 53 | | |
| 54 | + | |
46 | 55 | | |
47 | 56 | | |
48 | 57 | | |
49 | 58 | | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
50 | 62 | | |
51 | 63 | | |
52 | 64 | | |
| |||
89 | 101 | | |
90 | 102 | | |
91 | 103 | | |
92 | | - | |
93 | | - | |
94 | | - | |
| 104 | + | |
| 105 | + | |
95 | 106 | | |
96 | 107 | | |
97 | 108 | | |
| |||
105 | 116 | | |
106 | 117 | | |
107 | 118 | | |
108 | | - | |
109 | | - | |
| 119 | + | |
110 | 120 | | |
111 | | - | |
112 | | - | |
113 | | - | |
114 | | - | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
115 | 124 | | |
116 | | - | |
| 125 | + | |
117 | 126 | | |
118 | 127 | | |
119 | | - | |
| 128 | + | |
120 | 129 | | |
| 130 | + | |
| 131 | + | |
121 | 132 | | |
122 | 133 | | |
123 | 134 | | |
124 | 135 | | |
125 | 136 | | |
126 | 137 | | |
127 | | - | |
| 138 | + | |
128 | 139 | | |
129 | 140 | | |
130 | 141 | | |
131 | 142 | | |
132 | 143 | | |
133 | 144 | | |
134 | 145 | | |
135 | | - | |
| 146 | + | |
136 | 147 | | |
137 | 148 | | |
138 | 149 | | |
139 | 150 | | |
140 | 151 | | |
141 | | - | |
| 152 | + | |
142 | 153 | | |
143 | 154 | | |
144 | 155 | | |
| |||
230 | 241 | | |
231 | 242 | | |
232 | 243 | | |
233 | | - | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
234 | 253 | | |
235 | 254 | | |
236 | 255 | | |
| |||
304 | 323 | | |
305 | 324 | | |
306 | 325 | | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
0 commit comments