You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit ec97d1c
Browse filesBrowse the repository at this point in the historyBrowse files
## Which issue does this PR close?
- closes#11900
## Rationale for this change
When a query has `ORDER BY <cols> LIMIT N` on top of an outer join and
all sort columns come from the preserved side,
DataFusion currently runs the full join first, then sorts and limits. We
can push a copy of the `Sort(fetch=N)` to the preserved input, reducing
the number of rows entering the join.
**Before:**
```
Sort: t1.b ASC, fetch=3
Left Join: t1.a = t2.a
Scan: t1 ← scans ALL rows
Scan: t2
```
**After:**
```
Sort: t1.b ASC, fetch=3
Left Join: t1.a = t2.a
Sort: t1.b ASC, fetch=3 ← pushed down
Scan: t1 ← only top-3 rows enter join
Scan: t2
```
## What changes are included in this PR?
A new logical optimizer rule `PushDownTopKThroughJoin` that:
1. Matches `Sort` with `fetch = Some(N)` (TopK)
2. Looks through an optional `Projection` to find a `Join`
3. Checks join type is LEFT or RIGHT with no non-equijoin filter
4. Verifies all sort expression columns come from the preserved side
5. Inserts a copy of the `Sort(fetch=N)` on the preserved child
6. Keeps the top-level sort for correctness
## Are these changes tested?
Yes through UT
## Are there any user-facing changes?
No API changes.
---------
Co-authored-by: Subham Singhal <[email protected]>
Co-authored-by: Dmitrii Blaginin <[email protected]>
Co-authored-by: Kumar Ujjawal <[email protected]>
| 15 |`propagate_empty_relation`| Pushes empty-relation knowledge upward so operators fed by no rows collapse early. |
55
+
| 16 |`filter_null_join_keys`| Adds `IS NOT NULL` filters to nullable equijoin keys that can never match. |
56
+
| 17 |`eliminate_outer_join`| Rewrites outer joins to inner joins when later filters reject the NULL-extended rows. |
57
+
| 18 |`push_down_limit`| Moves literal limits to scans/unions, merges limits, and pushes `Sort(fetch=N)` onto a preserved join side only when every sort key is from it; the outer Sort stays.|
58
+
| 19 |`push_down_filter`| Moves filters as early as possible through filter-commutative operators. |
0 commit comments