Repository navigation
Commit d80703a
Ravel Fleet Executor
docs(sql): describe CompactSlicesExec and the slice charge it removes
docs/query-engine.md gains a paragraph after the one on DataFusion 55.2's
migrated aggregate streams: a slice of an aggregate's materialized output
reports the whole batch's buffers, RepartitionExec and SortExec reserve
that figure per buffered slice (apache/datafusion#24704), and the
CompactSlices rule inserts a CompactSlicesExec above every aggregate whose
output reaches one of them. It states where the exec sits on the 55.2 q29
plan, what its stream copies and what it passes through, its cost of one
copy of each such aggregate output, the metrics that report it, which edge
the integration test measures and which is pinned by placement alone, and
that there is no configuration knob.
changelog.d/2720.fixed.md records the operator-facing change: a grouped
aggregate with many groups over more than one SQL partition no longer
refuses with "the query memory pool is full and spill is disabled" on
DataFusion 55.2, with the ClickBench q29 figures that found it.
Refs: #2720
Signed-off-by: Ravel Fleet Executor <[email protected]>1 parent 7791e93 commit d80703a
2 files changed
Lines changed: 50 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1339 | 1339 | | |
1340 | 1340 | | |
1341 | 1341 | | |
| 1342 | + | |
| 1343 | + | |
| 1344 | + | |
| 1345 | + | |
| 1346 | + | |
| 1347 | + | |
| 1348 | + | |
| 1349 | + | |
| 1350 | + | |
| 1351 | + | |
| 1352 | + | |
| 1353 | + | |
| 1354 | + | |
| 1355 | + | |
| 1356 | + | |
| 1357 | + | |
| 1358 | + | |
| 1359 | + | |
| 1360 | + | |
| 1361 | + | |
| 1362 | + | |
| 1363 | + | |
| 1364 | + | |
| 1365 | + | |
| 1366 | + | |
| 1367 | + | |
| 1368 | + | |
| 1369 | + | |
| 1370 | + | |
| 1371 | + | |
| 1372 | + | |
| 1373 | + | |
| 1374 | + | |
| 1375 | + | |
| 1376 | + | |
1342 | 1377 | | |
1343 | 1378 | | |
1344 | 1379 | | |
| |||
0 commit comments