Repository navigation
Commit c1f0d54
authored
perf: avoid unnecessary large allocations (apache#22558)
## Which issue does this PR close?
Related a bit to apache#22526
~Needs rebasing once apache#22416 is
merged~
## Rationale for this change
split_off does this:
> Returns a newly allocated vector containing the elements in the range
[at, len). After the call, the original vector will be left containing
the elements [0, at) with its previous capacity unchanged.
which is bad when taking a small slice from a large Vec, for two
reasons:
* it will allocate memory for the remaining elements, which are a lot
more than n
* it will return a Vec with a very large capacity compared to its length
split_vec_min_alloc still has some issues:
apache#22548 but it uses drain +
collect when n is small, which is better because it only allocates for
the initial n elements and doesn't inflate the capacity
## What changes are included in this PR?
## Are these changes tested?
Yes
## Are there any user-facing changes?
No1 parent 7b78f0c commit c1f0d54
2 files changed
Lines changed: 102 additions & 11 deletions
File tree
- datafusion
- expr-common/src
- physical-plan/src/aggregates/group_values/single_group_by
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
| 48 | + | |
55 | 49 | | |
56 | 50 | | |
57 | 51 | | |
| |||
254 | 248 | | |
255 | 249 | | |
256 | 250 | | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| 27 | + | |
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
| |||
207 | 208 | | |
208 | 209 | | |
209 | 210 | | |
210 | | - | |
211 | | - | |
212 | | - | |
| 211 | + | |
213 | 212 | | |
214 | 213 | | |
215 | 214 | | |
| |||
223 | 222 | | |
224 | 223 | | |
225 | 224 | | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
0 commit comments