Repository navigation
Commit 1b857a3
fix: pick up a limit below a window the rule already grew past
Review feedback: one shape of #26659 was still left behind. A window the
rule *can* grow past leaves the walk in `Phase::Apply`, and the guard on
`get_limit` skipped every limit while in that phase. So on the way down to
an inner window, the inner `GlobalLimitExec` was never read; the walk fell
through to the cardinality check and dropped the chain at it instead, and
the inner sort stayed a full sort although a `LIMIT 5` sat right above it.
The phase only has to hold us back at a `SortPreservingMergeExec`, which
is both a limit source and an apply target -- reading its fetch while
carrying a limit down would overwrite the limit with it. A plain limit node
is never an apply target, so reading one is always safe, and it is more
current than the chain we were on, which has not found a sort to apply to.
The guard now covers just the one node that needs it.
`reset_limit` returns the phase to `FindOrGrow` rather than `optimize()`
doing it at the call site, which is the one difference from the patch in
the review. Starting a chain and abandoning the previous one are the same
event, `reset_limit` only ever runs when `get_limit` found a limit, and the
doc comment there promised the opposite and had to change either way. Happy
to move it back to the call site if you prefer it there.
The new case is the previous query with the outer frame changed from RANGE
to ROWS, and the change of frame is what makes it fail: a frame the rule
cannot grow past resets the state on the way down, so the limit below is
seen afterwards, while a frame it can grow past does not. Both levels order
by the same column, so `EnforceSorting` elides the outer sort and there is
no `apply_limit` in between to reset the state either.
Co-Authored-By: Claude Opus 5 <[email protected]>1 parent 4877863 commit 1b857a3
2 files changed
Lines changed: 72 additions & 4 deletions
File tree
- datafusion
- physical-optimizer/src
- sqllogictest/test_files
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | | - | |
59 | | - | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
60 | 61 | | |
61 | 62 | | |
62 | 63 | | |
| 64 | + | |
63 | 65 | | |
64 | 66 | | |
65 | 67 | | |
| |||
93 | 95 | | |
94 | 96 | | |
95 | 97 | | |
96 | | - | |
97 | | - | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
98 | 106 | | |
99 | 107 | | |
100 | 108 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1118 | 1118 | | |
1119 | 1119 | | |
1120 | 1120 | | |
| 1121 | + | |
| 1122 | + | |
| 1123 | + | |
| 1124 | + | |
| 1125 | + | |
| 1126 | + | |
| 1127 | + | |
| 1128 | + | |
| 1129 | + | |
| 1130 | + | |
| 1131 | + | |
| 1132 | + | |
| 1133 | + | |
| 1134 | + | |
| 1135 | + | |
| 1136 | + | |
| 1137 | + | |
| 1138 | + | |
| 1139 | + | |
| 1140 | + | |
| 1141 | + | |
| 1142 | + | |
| 1143 | + | |
| 1144 | + | |
| 1145 | + | |
| 1146 | + | |
| 1147 | + | |
| 1148 | + | |
| 1149 | + | |
| 1150 | + | |
| 1151 | + | |
| 1152 | + | |
| 1153 | + | |
| 1154 | + | |
| 1155 | + | |
| 1156 | + | |
| 1157 | + | |
| 1158 | + | |
| 1159 | + | |
| 1160 | + | |
| 1161 | + | |
| 1162 | + | |
| 1163 | + | |
| 1164 | + | |
| 1165 | + | |
| 1166 | + | |
| 1167 | + | |
| 1168 | + | |
| 1169 | + | |
| 1170 | + | |
| 1171 | + | |
| 1172 | + | |
| 1173 | + | |
| 1174 | + | |
| 1175 | + | |
| 1176 | + | |
| 1177 | + | |
| 1178 | + | |
| 1179 | + | |
| 1180 | + | |
0 commit comments