Repository navigation
Commit 1670bee
test: add sqllogictest coverage for DISTINCT / GROUP BY / aggregation on map columns (apache#23406)
## Which issue does this PR close?
- Closes apache#15428.
## Rationale for this change
`SELECT DISTINCT` on map columns used to fail with
`ArrowError(NotYetImplemented)` because arrow-rs's `RowConverter` did
not support Map types. That support has since landed upstream
(apache/arrow-rs#7879) and is available on DataFusion main via arrow
59.1, so these queries now work. As suggested in
apache#15428 (comment)
(maintainer comment), this PR adds sqllogictest coverage to lock in the
behavior and close the issue.
## What changes are included in this PR?
Adds a new test section to `datafusion/sqllogictest/test_files/map.slt`
covering:
- `SELECT DISTINCT` on a map column, including the exact reproducer from
the issue (`DISTINCT ... LIMIT`), collapsing of duplicate maps and
duplicate NULLs, and `DISTINCT` over map + scalar columns together
- `GROUP BY` on a map column, `GROUP BY` map + scalar keys, and `HAVING`
with a map grouping key
- Aggregations with map inputs: `COUNT(map)`, `COUNT(DISTINCT map)`,
`SUM` grouped by map, and `array_agg` of map values
- Edge cases: empty maps under `DISTINCT` (equal to each other, distinct
from `NULL`), NULL maps, and unsorted maps with the same entries in
different order being treated as distinct values
- `UNION` (distinct) on map columns
- `DISTINCT` / `GROUP BY` over the existing 209-row
`parquet_map.parquet` file for coverage on real Parquet-backed map data
Note: a bare `MAP {'key': NULL}` literal infers a `Null` value type and
fails type coercion across a VALUES list against `Map(Utf8, Int64)`
rows, so the test uses `CAST(NULL AS BIGINT)` for the null map value.
That coercion gap may be worth a separate follow-up issue.
## Are these changes tested?
Yes, this PR is test-only. All tests pass locally via `cargo test -p
datafusion-sqllogictest --test sqllogictests -- map.slt`.
## Are there any user-facing changes?
No. Test-only change; no API or behavior changes.1 parent 0020e1c commit 1670bee
1 file changed
Lines changed: 132 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
916 | 916 | | |
917 | 917 | | |
918 | 918 | | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
| 937 | + | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| 965 | + | |
| 966 | + | |
| 967 | + | |
| 968 | + | |
| 969 | + | |
| 970 | + | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
| 978 | + | |
| 979 | + | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
| 984 | + | |
| 985 | + | |
| 986 | + | |
| 987 | + | |
| 988 | + | |
| 989 | + | |
| 990 | + | |
| 991 | + | |
| 992 | + | |
| 993 | + | |
| 994 | + | |
| 995 | + | |
| 996 | + | |
| 997 | + | |
| 998 | + | |
| 999 | + | |
| 1000 | + | |
| 1001 | + | |
| 1002 | + | |
| 1003 | + | |
| 1004 | + | |
| 1005 | + | |
| 1006 | + | |
| 1007 | + | |
| 1008 | + | |
| 1009 | + | |
| 1010 | + | |
| 1011 | + | |
| 1012 | + | |
| 1013 | + | |
| 1014 | + | |
| 1015 | + | |
| 1016 | + | |
| 1017 | + | |
| 1018 | + | |
| 1019 | + | |
| 1020 | + | |
| 1021 | + | |
| 1022 | + | |
| 1023 | + | |
| 1024 | + | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
| 1030 | + | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
| 1042 | + | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
| 1048 | + | |
| 1049 | + | |
| 1050 | + | |
0 commit comments