Skip to content

Commit 7f0c2d2

Browse files
committed
test: cover grouping set equivalences and constants
On DataFusion 53.1.0 and 55.2.0 with target_partitions = 1, a GROUPING SETS ((a, b), (a)) query over rows filtered by a = b returned rows in the wrong ORDER BY order. The grouping-set AggregateExec projected the input equivalence a = b to its output, but the (a) set outputs b as NULL, so a = b is false on those rows. EnforceSorting used the equivalence and rewrote the sort [a ASC NULLS LAST, b ASC NULLS FIRST] to [b ASC NULLS LAST]. A filter constant such as b = 1 fails the same way. apache#24422 fixed this on main by building the properties of a grouping-set aggregate from empty EquivalenceProperties. Its tests check that grouping sets do not pick ordered aggregation. Add cases to grouping.slt for the equivalence, the constant, a row_number() variant and the query from apache#16965, in the target_partitions = 1 block. Closes apache#16965.
1 parent e4f9b03 commit 7f0c2d2

1 file changed

Lines changed: 64 additions & 0 deletions

File tree

‎datafusion/sqllogictest/test_files/grouping.slt‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,70 @@ LIMIT 10;
123123
statement ok
124124
RESET datafusion.execution.enable_migration_aggregate;
125125

126+
# a = b holds before the aggregate, but not after it: the (a) grouping set
127+
# outputs b as NULL.
128+
# (a, b) set: (1, 1), (2, 2)
129+
# (a) set: (1, NULL), (2, NULL)
130+
# So the sort must keep both a and b. Queries from PostgreSQL's groupingsets.sql.
131+
query TT
132+
EXPLAIN SELECT a, b FROM (VALUES (1, 1), (2, 2)) AS t (a, b) WHERE a = b
133+
GROUP BY GROUPING SETS ((a, b), (a)) ORDER BY a, b NULLS FIRST;
134+
----
135+
logical_plan
136+
01)Sort: t.a ASC NULLS LAST, t.b ASC NULLS FIRST
137+
02)--Projection: t.a, t.b
138+
03)----Aggregate: groupBy=[[GROUPING SETS ((t.a, t.b), (t.a))]], aggr=[[]]
139+
04)------SubqueryAlias: t
140+
05)--------Projection: column1 AS a, column2 AS b
141+
06)----------Filter: column2 = column1
142+
07)------------Values: (Int64(1), Int64(1)), (Int64(2), Int64(2))
143+
physical_plan
144+
01)SortExec: expr=[a@0 ASC NULLS LAST, b@1 ASC], preserve_partitioning=[false]
145+
02)--ProjectionExec: expr=[a@0 as a, b@1 as b]
146+
03)----AggregateExec: mode=Final, gby=[a@0 as a, b@1 as b, __grouping_id@2 as __grouping_id], aggr=[]
147+
04)------AggregateExec: mode=Partial, gby=[(a@0 as a, b@1 as b), (a@0 as a, NULL as b)], aggr=[]
148+
05)--------ProjectionExec: expr=[column1@0 as a, column2@1 as b]
149+
06)----------FilterExec: column2@1 = column1@0
150+
07)------------DataSourceExec: partitions=1, partition_sizes=[1]
151+
152+
query II
153+
SELECT a, b FROM (VALUES (1, 1), (2, 2)) AS t (a, b) WHERE a = b
154+
GROUP BY GROUPING SETS ((a, b), (a)) ORDER BY a, b NULLS FIRST;
155+
----
156+
1 NULL
157+
1 1
158+
2 NULL
159+
2 2
160+
161+
query III
162+
SELECT a, b, row_number() OVER (ORDER BY a, b NULLS FIRST)
163+
FROM (VALUES (1, 1), (2, 2)) AS t (a, b) WHERE a = b
164+
GROUP BY GROUPING SETS ((a, b), (a)) ORDER BY a, b NULLS FIRST;
165+
----
166+
1 NULL 1
167+
1 1 2
168+
2 NULL 3
169+
2 2 4
170+
171+
# Same for b = 1: the (a) set outputs (1, NULL), (2, NULL), so b stays in the sort.
172+
query II
173+
SELECT a, b FROM (VALUES (1, 1), (2, 1)) AS t (a, b) WHERE b = 1
174+
GROUP BY GROUPING SETS ((a, b), (a)) ORDER BY a, b NULLS FIRST;
175+
----
176+
1 NULL
177+
1 1
178+
2 NULL
179+
2 1
180+
181+
# https://github.com/apache/datafusion/issues/16965: a constant, sorted input
182+
# must still produce one row per grouping set.
183+
query T rowsort
184+
SELECT id FROM (SELECT 'id' AS id UNION ALL SELECT 'id' AS id ORDER BY id)
185+
GROUP BY GROUPING SETS ((id), ());
186+
----
187+
NULL
188+
id
189+
126190
statement ok
127191
SET datafusion.execution.target_partitions = 4;
128192

0 commit comments

Comments
 (0)