Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
review: make comments more precise; extract util
  • Loading branch information
2010YOUY01 committed Oct 7, 2026
commit c8986fdd80fc65a677f66c242917ec3b1112b30a
37 changes: 19 additions & 18 deletions datafusion/physical-optimizer/src/limited_distinct_aggregation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

use std::sync::Arc;

use datafusion_physical_plan::aggregates::{AggregateExec, AggregateMode};
use datafusion_physical_plan::aggregates::AggregateExec;
use datafusion_physical_plan::limit::{GlobalLimitExec, LocalLimitExec};
use datafusion_physical_plan::{
ChildrenPropertiesMode, ExecutionPlan, ExecutionPlanProperties,
Expand Down Expand Up @@ -60,23 +60,28 @@ use crate::PhysicalOptimizerRule;
/// Scan
/// ```
///
/// # Invariants before and after the rewrite
/// # What this rule assumes

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 like the wording

///
/// ## Before this rule
///
/// The default physical planner produces two adjacent aggregate stages with
/// compatible grouping expressions. This rule runs before other rules can combine
/// them into a `Single` aggregate or insert operators between them.
/// This rule assumes the logical aggregate only have one shape showed below, this
/// is what the current physical planning produces.
///
/// ```txt
/// AggregateExec(mode=Final)
/// AggregateExec(mode=Partial)
/// Limit
/// AggregateExec(mode=Final)
/// AggregateExec(mode=Partial)
/// ```
///
/// ## After this rule
/// If future changes or extensions produce a different shape, this rule skips the
/// rewrite rather than reporting an error, potentially missing an optimization
/// opportunity.
///
/// # What this rule promises
///
/// Immediately after an eligible rewrite, both stages have a soft-limit hint.
/// Otherwise, this rule leaves both stages unchanged.
///
/// For an eligible pair, both stages receive a soft-limit hint. Otherwise, both
/// stages are left unchanged.
/// If a later rule removes the limit, it won't affect correctness, but it may
/// miss an optimization opportunity.
#[derive(Debug)]
pub struct LimitedDistinctAggregation {}

Expand Down Expand Up @@ -122,12 +127,8 @@ impl LimitedDistinctAggregation {
};

// Step 2: Verify partial/final aggregate is compatible, then apply optimization
match (final_agg.mode(), partial_agg.mode()) {
(
AggregateMode::Final | AggregateMode::FinalPartitioned,
AggregateMode::Partial,
) if final_agg.group_expr() == &partial_agg.group_expr().as_final() => {}
_ => return Ok(Transformed::no(plan)),
if !final_agg.matches_partial(partial_agg) {
return Ok(Transformed::no(plan));
}

// Validate both stages before replacing either one.
Expand Down
17 changes: 17 additions & 0 deletions datafusion/physical-plan/src/aggregates/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1393,6 +1393,23 @@ impl AggregateExec {
&self.mode
}

/// Returns whether this is a compatible partial/final aggregation pair.
///
/// # Public Only for Internal Use:
///
/// This is made public for the physical optimizer to use, this is not part of
/// the public API.
#[doc(hidden)]
pub fn matches_partial(&self, partial: &Self) -> bool {
matches!(
(self.mode, partial.mode),
(
AggregateMode::Final | AggregateMode::FinalPartitioned,
AggregateMode::Partial,
)
) && self.group_expr() == &partial.group_expr().as_final()
}

/// Set a legacy limit hint. Unsupported requests leave ordinary aggregation.
#[deprecated(
since = "56.0.0",
Expand Down
Loading