Repository navigation
Commit 6e69fe9
fix(physical-optimizer): decline the WindowTopN rewrite when the FilterExec carries a fetch (apache#23599)
# Which issue does this PR close?
- Follow-up to apache#21479 (WindowTopN optimizer rule).
- Leaves apache#21596 (embedded projections) open — see the note at the
bottom.
# Rationale for this change
`WindowTopN` replaces the matched `FilterExec` outright, so anything the
filter carries beyond its predicate has to be reproduced in the new
plan, or the rule has to decline.
`FilterExec::fetch` is applied by `FilterExec::execute` *after* the
predicate. The rewritten plan has nowhere to put it: a
`PartitionedTopKExec` bounds rows **per partition**, which is not the
same as a row limit over the filtered output. Today the fetch is dropped
on the floor:
```
FilterExec: row_number@2 <= 3, fetch=1
BoundedWindowAggExec: ...
PlaceholderRowExec
```
is rewritten to
```
BoundedWindowAggExec: ...
PartitionedTopKExec: fn=row_number, fetch=3, partition=[pk@0], order=[val@1 ASC]
PlaceholderRowExec
```
— up to 3 rows *per partition* instead of 1 row overall.
**This is not reachable from the default rule list**, which runs
`LimitPushdown` (position 162) well after `WindowTopN` (position 117),
so no stock plan reaches the rule with a fetch set. The guard is cheap
insurance for pipelines that reorder the two, and it costs one
`is_some()` check rather than a new code path.
# What changes are included in this PR?
- `WindowTopN::try_transform` declines when `filter.fetch().is_some()`,
with a comment recording why the fetch cannot be carried and that the
default order makes this unreachable today.
# Are these changes tested?
Yes — `filter_with_fetch_is_declined` asserts the plan is left alone,
and first asserts that the *same fixture without the fetch* is
rewritten, so the test cannot pass because the shape was wrong. Removing
the guard fails it with the plan diff shown above.
# Are there any user-facing changes?
No. No stock plan reaches this path; the rule declines in one more case
for pipelines that do.
---
### Note on the original scope
This PR previously also taught `WindowTopN` to carry an embedded
`FilterExec` projection through the rewrite (apache#21596). Per the discussion
below with @2010YOUY01, that part is dropped: it is real logic rather
than a bail-out, and under the default rule order `ProjectionPushdown`
(142) runs *after* `WindowTopN` (117), so no e2e test can keep it
honest. apache#21596's premise — "this happens when `ProjectionPushdown` runs
before `WindowTopN`" — does not hold for the default list, and I have
noted that on the issue so the next attempt starts from the right
assumption.
Co-authored-by: Andrew Lamb <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>1 parent bcd205e commit 6e69fe9
2 files changed
Lines changed: 45 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
775 | 775 | | |
776 | 776 | | |
777 | 777 | | |
| 778 | + | |
| 779 | + | |
| 780 | + | |
| 781 | + | |
| 782 | + | |
| 783 | + | |
| 784 | + | |
| 785 | + | |
| 786 | + | |
| 787 | + | |
| 788 | + | |
| 789 | + | |
| 790 | + | |
| 791 | + | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
| 806 | + | |
| 807 | + | |
778 | 808 | | |
779 | 809 | | |
780 | 810 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
162 | 162 | | |
163 | 163 | | |
164 | 164 | | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
165 | 180 | | |
166 | 181 | | |
167 | 182 | | |
| |||
0 commit comments