You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 730a9fe
Browse filesBrowse the repository at this point in the historyBrowse files
fix: treat a deterministic computed GROUP BY expression as determined by a key
A computed GROUP BY expression was never counted as determined, so a key
next to one, such as `GROUP BY i_item_sk, upper(i_desc)`, was no longer a
key of the aggregate output, and queries that plan on 55.2.0 failed.
The new `aggregate_functional_dependencies_with_computed_inputs` takes a
`computed_input_indices` parameter with the input fields each
deterministic computed GROUP BY expression reads. Such an expression is
determined when every field it reads is. Volatile expressions and
expressions with subqueries or outer references are never determined.
`aggregate_functional_dependencies` keeps its signature and is deprecated
since 57.0.0; it treats every computed expression as not determined.
Also adds tests for the DISTINCT and window frame symptoms of apache#27112.
Copy file name to clipboardExpand all lines: datafusion/sqllogictest/test_files/functional_dependencies.slt
+60-1Lines changed: 60 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -787,7 +787,23 @@ ORDER BY i_item_sk;
787
787
1 1
788
788
2 1
789
789
790
-
# 8.4 The planner adds `i_desc`, which `i_item_sk` determines, to the inner
790
+
# 8.4 `i_item_sk` is not a key of `item_dates`: the DISTINCT must stay, and the
791
+
# default window frame must stay RANGE (peers of item 1 count together).
792
+
query I
793
+
SELECT DISTINCT i_item_sk FROM item_dates ORDER BY i_item_sk;
794
+
----
795
+
1
796
+
2
797
+
798
+
query II
799
+
SELECT i_item_sk, count(*) OVER (ORDER BY i_item_sk) FROM item_dates
800
+
ORDER BY i_item_sk;
801
+
----
802
+
1 2
803
+
1 2
804
+
2 3
805
+
806
+
# 8.5 The planner adds `i_desc`, which `i_item_sk` determines, to the inner
791
807
# GROUP BY. `i_desc` is an extraneous column of the inner aggregate's key and
792
808
# is removed from it, so the key still exists after the projection removes
793
809
# `i_desc`, and `c` can be selected.
@@ -801,6 +817,49 @@ GROUP BY i_item_sk, s_date ORDER BY i_item_sk, s_date;
801
817
1 101 1
802
818
2 100 1
803
819
820
+
# 8.6 `upper(i_desc)` reads only `i_desc`, which `i_item_sk` determines, so
821
+
# `i_item_sk` is a key of the inner aggregate and `c` can be selected.
822
+
query II
823
+
SELECT i_item_sk, c
824
+
FROM (SELECT i_item_sk, upper(i_desc) AS u, count(*) AS c
825
+
FROM item JOIN sales ON i_item_sk = s_item_sk
826
+
GROUP BY i_item_sk, upper(i_desc)) q
827
+
GROUP BY i_item_sk ORDER BY i_item_sk;
828
+
----
829
+
1 2
830
+
2 1
831
+
832
+
# 8.7 `s_date + 1` reads `s_date`, which `i_item_sk` does not determine.
833
+
statement error
834
+
SELECT i_item_sk, c
835
+
FROM (SELECT i_item_sk, s_date + 1 AS d, count(*) AS c
836
+
FROM item JOIN sales ON i_item_sk = s_item_sk
837
+
GROUP BY i_item_sk, s_date + 1) q
838
+
GROUP BY i_item_sk;
839
+
----
840
+
DataFusion error: Error during planning: Column in SELECT must be in GROUP BY or an aggregate function: While expanding wildcard, column "q.c" must appear in the GROUP BY clause or must be part of an aggregate function, currently only "q.i_item_sk" appears in the SELECT clause satisfies this requirement
841
+
842
+
843
+
# 8.8 A volatile expression is never determined: the join stays an inner join.
844
+
query TT
845
+
EXPLAIN SELECT sum(o_amount) FROM orders
846
+
JOIN (SELECT i_item_sk FROM item JOIN sales ON i_item_sk = s_item_sk
847
+
GROUP BY i_item_sk, random() > 0.5) g
848
+
ON o_item_sk = g.i_item_sk;
849
+
----
850
+
logical_plan
851
+
01)Aggregate: groupBy=[[]], aggr=[[sum(CAST(orders.o_amount AS Int64))]]
0 commit comments