You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 9358dcd
Browse filesBrowse the repository at this point in the historyBrowse files
fix: preserve aggregate filter pushdown order (apache#22926)
## Which issue does this PR close?
- Closesapache#22925.
## Rationale for this change
`FilterPushdown` maps each child pushdown result back to its original
parent
filter by position. `AggregateExec::gather_filters_for_pushdown`
previously
split parent filters into safe and unsafe buckets and then concatenated
the
results, which changed their order.
For example, in `cnt@2 = 1 AND b@1 = bar`, the grouping-column predicate
on
`b` can cross the aggregate, while the predicate on aggregate output
`cnt`
must remain above it. Reordering the returned results could make the
optimizer
associate the pushed-down `b` result with the `cnt` predicate and
incorrectly
remove the latter.
Aggregate pushdown also needs to account for empty-input semantics. A
global
aggregate, or a grouping-sets aggregate containing `()`, can emit a row
even
when its input is empty. Moving any parent predicate below such an
aggregate —
including a column-free predicate such as `false` — can therefore change
the
result.
## What changes are included in this PR?
- Preserve parent-filter order by constructing each child description
once
with `ChildFilterDescription::from_child_with_allowed_indices`.
- Allow only grouping-output columns that are present in every grouping
set;
aggregate-result columns remain above the aggregate.
- Mark all parent filters unsupported for global aggregates and grouping
sets
containing an empty grouping set, while retaining aggregate-generated
dynamic filters.
- Validate that every child returns one parent-filter result per input
filter
before positional remapping.
- Add physical optimizer and SQL regression coverage for mixed filter
order,
global-aggregate name collisions, constant predicates, and grouping
sets.
## Are these changes tested?
Yes:
- `cargo test -p datafusion --test core_integration
physical_optimizer::filter_pushdown`
- `cargo fmt --all -- --check`
- `cargo check -p datafusion-physical-plan -p
datafusion-physical-optimizer`
- `git diff --check upstream/main`
- Full GitHub CI, including Rust tests, clippy, and sqllogictests
## Are there any user-facing changes?
There are no public API changes. Filter pushdown now preserves mixed
aggregate
predicates correctly and avoids moving predicates across aggregates when
doing
so could change empty-input results.
0 commit comments