Repository navigation
Commit 6f1436f
authored
fix: compact
## Which issue does this PR close?
Closes apache#26744.
## Rationale for this change
`DISTINCT ON` queries carrying nested payloads can fail with a memory
reservation error during aggregate spill/replay because merged
`FIRST_VALUE` states retain backing buffers from larger arrays.
## What changes are included in this PR?
Call `self.first.compact()` after extracting the selected scalar in
`TrivialFirstValueAccumulator::merge_batch`, matching the existing
behavior in `update_batch`.
## What is the testing strategy for this PR?
Locally validated against unpatched and patched DataFusion 55.2.0. An
accumulator regression check confirms that merging a list state
preserves its value and reports the same memory size as updating it
directly; it fails before the fix and passes afterward.
The spill workload described in apache#26744 fails before the fix and returns
all 8,192 rows with correct nested payloads afterward, while still
exercising spilling.
## Are there any user-facing changes?
No.FIRST_VALUE scalars during merge (apache#26743)1 parent 97ff724 commit 6f1436f
2 files changed
Lines changed: 47 additions & 0 deletions
File tree
- datafusion
- functions-aggregate/src
- sqllogictest/test_files
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
977 | 977 | | |
978 | 978 | | |
979 | 979 | | |
| 980 | + | |
980 | 981 | | |
981 | 982 | | |
982 | 983 | | |
| |||
2972 | 2973 | | |
2973 | 2974 | | |
2974 | 2975 | | |
| 2976 | + | |
| 2977 | + | |
| 2978 | + | |
| 2979 | + | |
| 2980 | + | |
| 2981 | + | |
| 2982 | + | |
| 2983 | + | |
| 2984 | + | |
| 2985 | + | |
| 2986 | + | |
| 2987 | + | |
| 2988 | + | |
| 2989 | + | |
| 2990 | + | |
| 2991 | + | |
| 2992 | + | |
| 2993 | + | |
| 2994 | + | |
| 2995 | + | |
| 2996 | + | |
| 2997 | + | |
2975 | 2998 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
239 | 239 | | |
240 | 240 | | |
241 | 241 | | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
242 | 266 | | |
243 | 267 | | |
244 | 268 | | |
| |||
0 commit comments