Skip to content

Commit 2e83c93

Browse files
zhuqi-lucasxudong963
authored andcommitted
Skip unnecessary plan rebuild in adjust_input_keys_ordering for non-join plans (apache#21947)
Closes apache#21946 `adjust_input_keys_ordering` returns `Transformed::yes` unconditionally in the default else branch, even when `requirements.data` is empty and no changes were made. This triggers unnecessary `with_new_children` rebuilds on every node in the plan tree for non-join/non-aggregate queries. For plans with custom `ExecutionPlan` nodes whose `with_new_children` is expensive (e.g. nodes that re-evaluate cost functions on rebuild), this causes significant overhead. Add an early return with `Transformed::no` when `requirements.data.is_empty()` in the default else branch of `adjust_input_keys_ordering`. This skips the unnecessary plan tree rebuild for simple scan/filter/limit plans that have no join key reordering requirements. Yes, two unit tests added: - `adjust_input_keys_ordering_no_transform_for_scan` — verifies a bare parquet scan returns `Transformed::no` - `adjust_input_keys_ordering_no_transform_for_filter_scan` — verifies a filter→scan tree returns `Transformed::no` via `transform_down` No. This is a performance optimization that does not change query results or plan structure. (cherry picked from commit 05a6c45)
1 parent f069fa6 commit 2e83c93

2 files changed

Lines changed: 33 additions & 0 deletions

File tree

‎datafusion/core/tests/physical_optimizer/enforce_distribution.rs‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3665,3 +3665,33 @@ fn test_replace_order_preserving_variants_with_fetch() -> Result<()> {
36653665

36663666
Ok(())
36673667
}
3668+
3669+
/// Verifies that `adjust_input_keys_ordering` returns `Transformed::no`
3670+
/// for a simple scan plan with no key requirements, avoiding an
3671+
/// unnecessary plan rebuild.
3672+
#[test]
3673+
fn adjust_input_keys_ordering_no_transform_for_scan() -> Result<()> {
3674+
let plan: Arc<dyn ExecutionPlan> = parquet_exec();
3675+
let requirements = PlanWithKeyRequirements::new_default(plan);
3676+
let result = adjust_input_keys_ordering(requirements)?;
3677+
assert!(
3678+
!result.transformed,
3679+
"expected Transformed::no for a scan plan with empty requirements"
3680+
);
3681+
Ok(())
3682+
}
3683+
3684+
/// Verifies that `adjust_input_keys_ordering` applied via `transform_down`
3685+
/// over a filter -> scan tree returns `Transformed::no` when there are no
3686+
/// join/aggregate key requirements.
3687+
#[test]
3688+
fn adjust_input_keys_ordering_no_transform_for_filter_scan() -> Result<()> {
3689+
let plan: Arc<dyn ExecutionPlan> = filter_exec(parquet_exec());
3690+
let requirements = PlanWithKeyRequirements::new_default(plan);
3691+
let result = requirements.transform_down(adjust_input_keys_ordering)?;
3692+
assert!(
3693+
!result.transformed,
3694+
"expected Transformed::no for a filter->scan tree with no key requirements"
3695+
);
3696+
Ok(())
3697+
}

‎datafusion/physical-optimizer/src/enforce_distribution.rs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -415,6 +415,9 @@ pub fn adjust_input_keys_ordering(
415415
|| plan.as_any().downcast_ref::<WindowAggExec>().is_some()
416416
{
417417
requirements.data.clear();
418+
} else if requirements.data.is_empty() {
419+
// No requirements to push down and no plan changes — skip rebuild.
420+
return Ok(Transformed::no(requirements));
418421
} else {
419422
// By default, push down the parent requirements to children
420423
for child in requirements.children.iter_mut() {

0 commit comments

Comments
 (0)