Repository navigation
Commit 4d28bde
committed
[fix](be) Reduce complex array_agg serialization memory
### What problem does this PR solve?
Issue Number: close apache#65976
Related PR: apache#66601
Problem Summary: Complex array_agg states stored immutable type and execution-version metadata in every group and foreach position, adding per-state overhead. Serialization also allocated an untracked upper-bound string and copied it into the tracked destination, doubling peak payload memory. Keep serializer metadata on the aggregate function, serialize once into tracked padded destination storage with rollback on failure, and retain exact logical framing for zero-copy reads.
### Release note
Reduce memory usage for complex array_agg and array_agg_foreach state serialization.
### Check List (For Author)
- Test: Unit Test
- Unit Test: GLIBC_COMPATIBILITY=OFF ./run-be-ut.sh -j 48 --run --filter=AggregateFunctionArrayAggTest.*:AggregateFunctionExceptionTest.*
- Behavior changed: Yes. Complex array_agg state serialization removes redundant per-state metadata and tracked scratch duplication without changing query results.
- Does this need documentation: No1 parent 6d36bea commit 4d28bde
2 files changed
Lines changed: 102 additions & 28 deletions
File tree
- be
- src/exprs/aggregate
- test/exprs/aggregate
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
41 | 42 | | |
42 | 43 | | |
43 | 44 | | |
| |||
137 | 138 | | |
138 | 139 | | |
139 | 140 | | |
| 141 | + | |
140 | 142 | | |
141 | 143 | | |
142 | 144 | | |
| |||
232 | 234 | | |
233 | 235 | | |
234 | 236 | | |
| 237 | + | |
235 | 238 | | |
236 | 239 | | |
237 | 240 | | |
238 | | - | |
239 | | - | |
240 | 241 | | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
| 242 | + | |
| 243 | + | |
245 | 244 | | |
246 | 245 | | |
247 | 246 | | |
| |||
267 | 266 | | |
268 | 267 | | |
269 | 268 | | |
270 | | - | |
271 | | - | |
272 | | - | |
273 | | - | |
274 | | - | |
275 | | - | |
276 | | - | |
277 | | - | |
278 | | - | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
279 | 287 | | |
280 | 288 | | |
281 | | - | |
| 289 | + | |
282 | 290 | | |
283 | 291 | | |
284 | | - | |
| 292 | + | |
285 | 293 | | |
286 | | - | |
| 294 | + | |
287 | 295 | | |
288 | 296 | | |
289 | 297 | | |
| |||
311 | 319 | | |
312 | 320 | | |
313 | 321 | | |
314 | | - | |
315 | | - | |
316 | | - | |
317 | | - | |
318 | | - | |
319 | | - | |
320 | | - | |
321 | | - | |
322 | 322 | | |
323 | 323 | | |
324 | 324 | | |
| |||
330 | 330 | | |
331 | 331 | | |
332 | 332 | | |
333 | | - | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
334 | 338 | | |
335 | 339 | | |
336 | 340 | | |
337 | 341 | | |
338 | | - | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
339 | 347 | | |
340 | 348 | | |
341 | 349 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| 47 | + | |
47 | 48 | | |
48 | 49 | | |
49 | 50 | | |
| 51 | + | |
| 52 | + | |
50 | 53 | | |
51 | 54 | | |
52 | 55 | | |
| |||
121 | 124 | | |
122 | 125 | | |
123 | 126 | | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
124 | 137 | | |
125 | 138 | | |
126 | 139 | | |
| |||
350 | 363 | | |
351 | 364 | | |
352 | 365 | | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
353 | 419 | | |
354 | 420 | | |
355 | 421 | | |
| |||
0 commit comments