Repository navigation
Commit 65f406f
committed
fix: declare the order of a native range for the joins above it
Spark's RangeExec reports id ascending, or descending for a negative step, and Spark drops a sort
above an outer hash join on that basis. The native range is a leaf with no JVM input, so the
planner declared no order on the join's probe input and the join moved its unmatched rows.
LazyMemoryExec::add_ordering leaves the cached output ordering of the plan properties stale in
DataFusion 55, so the planner wraps the range in a SortedInputExec instead, which now reports its
input's metrics.1 parent 8e75130 commit 65f406f
4 files changed
Lines changed: 52 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| |||
101 | 102 | | |
102 | 103 | | |
103 | 104 | | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
104 | 111 | | |
105 | 112 | | |
106 | 113 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
91 | | - | |
| 91 | + | |
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
| |||
2065 | 2065 | | |
2066 | 2066 | | |
2067 | 2067 | | |
2068 | | - | |
| 2068 | + | |
2069 | 2069 | | |
2070 | 2070 | | |
2071 | 2071 | | |
2072 | 2072 | | |
2073 | 2073 | | |
2074 | 2074 | | |
2075 | 2075 | | |
| 2076 | + | |
| 2077 | + | |
| 2078 | + | |
| 2079 | + | |
| 2080 | + | |
| 2081 | + | |
| 2082 | + | |
2076 | 2083 | | |
2077 | 2084 | | |
2078 | 2085 | | |
2079 | | - | |
| 2086 | + | |
2080 | 2087 | | |
2081 | 2088 | | |
2082 | 2089 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
40 | | - | |
| 40 | + | |
41 | 41 | | |
42 | 42 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
19 | 20 | | |
20 | 21 | | |
| 22 | + | |
| 23 | + | |
21 | 24 | | |
22 | 25 | | |
23 | 26 | | |
| |||
58 | 61 | | |
59 | 62 | | |
60 | 63 | | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
61 | 81 | | |
62 | 82 | | |
63 | 83 | | |
| |||
247 | 267 | | |
248 | 268 | | |
249 | 269 | | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
250 | 284 | | |
251 | 285 | | |
252 | 286 | | |
| |||
0 commit comments