Repository navigation
Commit 9c44fc0
authored
fix: count
## Which issue does this PR close?
- Closes #24735.
## Rationale for this change
`aggregation_time` on `GroupedHashAggregateStream` is added inside the
per-accumulator loop while `agg_start_time` is taken once before it, so
with N aggregate functions the k-th accumulator re-adds the elapsed time
of all k accumulators so far, inflating the metric by up to (N+1)/2. On
a 28-function GROUP BY a single task reported 4m22s of
`aggregation_time` inside a 2m0s stage. Single-aggregate plans are
unaffected. The newer `aggregate_hash_table` implementations already
scope the timer around the whole loop.
## What changes are included in this PR?
Move the `add_elapsed` call after the accumulator loop, once per
interned batch. `time_calculating_group_ids`, which shares
`agg_start_time` as its end point, is unchanged, and the per-accumulator
`aggregate_accumulator_metrics` timings are unaffected.
## Are these changes tested?
Covered by the existing aggregate tests (`cargo test -p
datafusion-physical-plan aggregates`, 193 passed). The metric's value is
wall-clock time, which the existing tests do not assert exactly; the
inflation itself was measured on a 28-aggregate workload where the fixed
metric now stays within the operator's elapsed time.
## Are there any user-facing changes?
`aggregation_time` (shown in `EXPLAIN ANALYZE` as elapsed_compute subset
time) now reports the actual time spent in accumulators on plans with
more than one aggregate function; previously it over-reported on such
plans.aggregation_time once per batch instead of once per accumulator (#24736)1 parent e4cf35c commit 9c44fc0
2 files changed
Lines changed: 69 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
475 | 475 | | |
476 | 476 | | |
477 | 477 | | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
478 | 543 | | |
479 | 544 | | |
480 | 545 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
978 | 978 | | |
979 | 979 | | |
980 | 980 | | |
981 | | - | |
982 | | - | |
983 | | - | |
984 | 981 | | |
| 982 | + | |
| 983 | + | |
| 984 | + | |
| 985 | + | |
985 | 986 | | |
986 | 987 | | |
987 | 988 | | |
| |||
0 commit comments