You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 6ff484a
Browse filesBrowse the repository at this point in the historyBrowse files
## Which issue does this PR close?
- Closesapache#25018
## Rationale for this change
A grouping column with one fixed value never changes, so it cannot tell
aggregation that an earlier group is complete.
Currently the logic uses such columns when choosing how aggregation
processes its input. This can select the ordered path even when the
remaining grouping columns are not ordered.
## What changes are included in this PR?
- Ignore fixed grouping columns when deciding when groups can be
completed.
- Preserve ordered aggregation when all grouping columns that can change
are ordered.
- Update relevant plans
## What is the testing strategy for this PR?
Added tests for a grouping column fixed by a filter. The tests fail
without the fix.
## Are there any user-facing changes?
No query results or public APIs change. DataFusion now chooses the
appropriate aggregation path for fixed grouping columns.
Copy file name to clipboardExpand all lines: datafusion/sqllogictest/test_files/aggregate.slt
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -8789,9 +8789,9 @@ logical_plan
8789
8789
15)------------EmptyRelation: rows=1
8790
8790
physical_plan
8791
8791
01)ProjectionExec: expr=[last_value(a.foo) ORDER BY [a.foo ASC NULLS LAST]@1 as last_value(a.foo) ORDER BY [a.foo ASC NULLS LAST], sum(DISTINCT Int64(1))@2 as sum(DISTINCT Int64(1))]
8792
-
02)--AggregateExec: mode=FinalPartitioned, gby=[id@0 as id], aggr=[last_value(a.foo) ORDER BY [a.foo ASC NULLS LAST], sum(DISTINCT Int64(1))], ordering_mode=Sorted
8792
+
02)--AggregateExec: mode=FinalPartitioned, gby=[id@0 as id], aggr=[last_value(a.foo) ORDER BY [a.foo ASC NULLS LAST], sum(DISTINCT Int64(1))]
04)------AggregateExec: mode=Partial, gby=[id@0 as id], aggr=[last_value(a.foo) ORDER BY [a.foo ASC NULLS LAST], sum(DISTINCT Int64(1))], ordering_mode=Sorted
8794
+
04)------AggregateExec: mode=Partial, gby=[id@0 as id], aggr=[last_value(a.foo) ORDER BY [a.foo ASC NULLS LAST], sum(DISTINCT Int64(1))]
8795
8795
05)--------UnionExec
8796
8796
06)----------ProjectionExec: expr=[1 as id, 2 as foo]
05)--------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/push_down_filter_regression/agg_filter_pushdown.parquet]]}, projection=[a, b], file_type=parquet, predicate=a@0 = x, pruning_predicate=a_null_count@2 != row_count@3 AND a_min@0 <= x AND x <= a_max@1, required_guarantees=[a in (x)]
0 commit comments