Repository navigation
Commit ef0a207
fix: keep probe-side constants in join equivalence properties (apache#26459)
- Closes apache#26283.
A query that orders by a column that is constant on the probe side of a
join fails to plan:
```text
SanityCheckPlan
caused by
Error during planning: Plan: ["SortPreservingMergeExec: [k@0 DESC]", " HashJoinExec: ..."] does not satisfy order requirements: [k@0 DESC]. Child-0 order: []
```
`EquivalenceClass::try_with_offset` builds a new class from the
expressions and does not copy `constant`. `EquivalenceGroup::join` uses
it for every class of the right side, so the join output forgets that
`k` is constant. `EnsureRequirements` removes the sort because `k` is
constant below the join, and `SanityCheckPlan` then rejects the plan
because the join output no longer says so.
- `try_with_offset` keeps the class's constant value.
- `EquivalenceGroup::join` clears the right side's constants for `Left`
and `Full` joins. Unmatched rows pad that side with NULLs, so a constant
there no longer holds. Without this, copying the constant would drop a
sort that is needed.
The first version of this PR added a sort in `EnsureRequirements`
wherever the requirement could not be proven. Following review, it now
fixes the join instead. The plan for apache#26283 then has no sort at all,
instead of a sort on a constant column.
The same problem exists the other way round: the left side's constants
are kept for `Right` and `Full` joins, where that side is padded with
NULLs too. For example, this returns `5` before the NULLs:
```sql
CREATE TABLE t(a INT) AS VALUES (1), (2), (3);
CREATE TABLE u(a INT) AS VALUES (9), (10), (1);
SELECT s.k, u.a FROM (SELECT a, 5 AS k FROM t) s RIGHT JOIN u ON s.a = u.a
ORDER BY s.k DESC NULLS FIRST;
```
This PR does not change the left side, because apache#26042 already rebuilds
the classes of the side that is padded with NULLs, which drops these
constants as well.
- `class.rs`: a unit test that a right-side constant stays constant
after `Inner` and `Right` joins, and not after `Left` and `Full` joins.
- `order.slt`: both queries from apache#26283, with results and `EXPLAIN`.
Without the change, both fail to plan.
- `order.slt`: a `LEFT JOIN` where the probe-side constant is padded
with NULLs. If the constant is not cleared, the sort on it is dropped
and the rows come back in the wrong order.
Queries like the one in apache#26283 plan and run instead of failing. There
are no API changes.
(cherry picked from commit bd219dc)1 parent a89b540 commit ef0a207
3 files changed
Lines changed: 187 additions & 4 deletions
File tree
- datafusion
- physical-expr/src/equivalence
- physical-plan/src/joins
- sqllogictest/test_files
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
245 | 245 | | |
246 | 246 | | |
247 | 247 | | |
248 | | - | |
| 248 | + | |
| 249 | + | |
249 | 250 | | |
250 | 251 | | |
251 | 252 | | |
| |||
256 | 257 | | |
257 | 258 | | |
258 | 259 | | |
| 260 | + | |
259 | 261 | | |
260 | 262 | | |
261 | 263 | | |
| |||
791 | 793 | | |
792 | 794 | | |
793 | 795 | | |
794 | | - | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
795 | 806 | | |
796 | 807 | | |
797 | 808 | | |
| |||
1245 | 1256 | | |
1246 | 1257 | | |
1247 | 1258 | | |
| 1259 | + | |
| 1260 | + | |
| 1261 | + | |
| 1262 | + | |
| 1263 | + | |
| 1264 | + | |
| 1265 | + | |
| 1266 | + | |
| 1267 | + | |
| 1268 | + | |
| 1269 | + | |
| 1270 | + | |
| 1271 | + | |
| 1272 | + | |
| 1273 | + | |
| 1274 | + | |
| 1275 | + | |
| 1276 | + | |
| 1277 | + | |
| 1278 | + | |
| 1279 | + | |
| 1280 | + | |
| 1281 | + | |
| 1282 | + | |
| 1283 | + | |
| 1284 | + | |
| 1285 | + | |
| 1286 | + | |
| 1287 | + | |
| 1288 | + | |
| 1289 | + | |
| 1290 | + | |
| 1291 | + | |
| 1292 | + | |
| 1293 | + | |
| 1294 | + | |
| 1295 | + | |
1248 | 1296 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
150 | 150 | | |
151 | 151 | | |
152 | 152 | | |
153 | | - | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
154 | 156 | | |
155 | 157 | | |
156 | 158 | | |
157 | 159 | | |
158 | 160 | | |
159 | | - | |
| 161 | + | |
160 | 162 | | |
161 | 163 | | |
162 | 164 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1969 | 1969 | | |
1970 | 1970 | | |
1971 | 1971 | | |
| 1972 | + | |
| 1973 | + | |
| 1974 | + | |
| 1975 | + | |
| 1976 | + | |
| 1977 | + | |
| 1978 | + | |
| 1979 | + | |
| 1980 | + | |
| 1981 | + | |
| 1982 | + | |
| 1983 | + | |
| 1984 | + | |
| 1985 | + | |
| 1986 | + | |
| 1987 | + | |
| 1988 | + | |
| 1989 | + | |
| 1990 | + | |
| 1991 | + | |
| 1992 | + | |
| 1993 | + | |
| 1994 | + | |
| 1995 | + | |
| 1996 | + | |
| 1997 | + | |
| 1998 | + | |
| 1999 | + | |
| 2000 | + | |
| 2001 | + | |
| 2002 | + | |
| 2003 | + | |
| 2004 | + | |
| 2005 | + | |
| 2006 | + | |
| 2007 | + | |
| 2008 | + | |
| 2009 | + | |
| 2010 | + | |
| 2011 | + | |
| 2012 | + | |
| 2013 | + | |
| 2014 | + | |
| 2015 | + | |
| 2016 | + | |
| 2017 | + | |
| 2018 | + | |
| 2019 | + | |
| 2020 | + | |
| 2021 | + | |
| 2022 | + | |
| 2023 | + | |
| 2024 | + | |
| 2025 | + | |
| 2026 | + | |
| 2027 | + | |
| 2028 | + | |
| 2029 | + | |
| 2030 | + | |
| 2031 | + | |
| 2032 | + | |
| 2033 | + | |
| 2034 | + | |
| 2035 | + | |
| 2036 | + | |
| 2037 | + | |
| 2038 | + | |
| 2039 | + | |
| 2040 | + | |
| 2041 | + | |
| 2042 | + | |
| 2043 | + | |
| 2044 | + | |
| 2045 | + | |
| 2046 | + | |
| 2047 | + | |
| 2048 | + | |
| 2049 | + | |
| 2050 | + | |
| 2051 | + | |
| 2052 | + | |
| 2053 | + | |
| 2054 | + | |
| 2055 | + | |
| 2056 | + | |
| 2057 | + | |
| 2058 | + | |
| 2059 | + | |
| 2060 | + | |
| 2061 | + | |
| 2062 | + | |
| 2063 | + | |
| 2064 | + | |
| 2065 | + | |
| 2066 | + | |
| 2067 | + | |
| 2068 | + | |
| 2069 | + | |
| 2070 | + | |
| 2071 | + | |
| 2072 | + | |
| 2073 | + | |
| 2074 | + | |
| 2075 | + | |
| 2076 | + | |
| 2077 | + | |
| 2078 | + | |
| 2079 | + | |
| 2080 | + | |
| 2081 | + | |
| 2082 | + | |
| 2083 | + | |
| 2084 | + | |
| 2085 | + | |
| 2086 | + | |
| 2087 | + | |
| 2088 | + | |
| 2089 | + | |
| 2090 | + | |
| 2091 | + | |
| 2092 | + | |
| 2093 | + | |
| 2094 | + | |
| 2095 | + | |
| 2096 | + | |
| 2097 | + | |
| 2098 | + | |
| 2099 | + | |
| 2100 | + | |
| 2101 | + | |
| 2102 | + | |
| 2103 | + | |
| 2104 | + | |
0 commit comments