Repository navigation
Commit f4c8ba1
authored
fix(proto): serialize Global/LocalLimitExec required_ordering (apache#24183)
## Which issue does this PR close?
- Closes apache#24173
## Rationale for this change
`GlobalLimitExec`/`LocalLimitExec` `required_ordering` is not
serialized, so a roundtrip loses the only record that a pushed-down
`LIMIT` is order-sensitive. Re-running `LimitPushdown` on the decoded
plan then sets `preserve_order: false` on the scan, which is free to
read files out of order — `ORDER BY ... LIMIT` can return the wrong
rows.
## What changes are included in this PR?
- Add `required_ordering` to `GlobalLimitExecNode` (field 4) and
`LocalLimitExecNode` (field 3); empty list means `None`
- New
`optional_ordering_try_to_proto`/`optional_ordering_try_from_proto`
helpers in `physical-expr-common`, also reused by
`SymmetricHashJoinExec` serde which hand-rolled the same pattern
- Preserve `required_ordering` in both execs' `with_new_children`
## Are these changes tested?
Yes new tests that would fail on main if not fixed
## Are there any user-facing changes?
No1 parent 0f68e23 commit f4c8ba1
9 files changed
Lines changed: 322 additions & 58 deletions
File tree
- datafusion
- core/tests/physical_optimizer
- physical-expr-common/src
- physical-optimizer/src/ensure_requirements/enforce_sorting
- physical-plan/src
- joins
- proto-models
- proto
- src/generated
- proto/tests/cases
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| 60 | + | |
| 61 | + | |
60 | 62 | | |
61 | 63 | | |
62 | 64 | | |
| |||
2297 | 2299 | | |
2298 | 2300 | | |
2299 | 2301 | | |
| 2302 | + | |
| 2303 | + | |
| 2304 | + | |
| 2305 | + | |
| 2306 | + | |
| 2307 | + | |
| 2308 | + | |
| 2309 | + | |
| 2310 | + | |
| 2311 | + | |
| 2312 | + | |
| 2313 | + | |
| 2314 | + | |
| 2315 | + | |
| 2316 | + | |
| 2317 | + | |
| 2318 | + | |
| 2319 | + | |
| 2320 | + | |
| 2321 | + | |
| 2322 | + | |
| 2323 | + | |
| 2324 | + | |
| 2325 | + | |
| 2326 | + | |
| 2327 | + | |
| 2328 | + | |
| 2329 | + | |
| 2330 | + | |
| 2331 | + | |
| 2332 | + | |
| 2333 | + | |
| 2334 | + | |
| 2335 | + | |
| 2336 | + | |
| 2337 | + | |
| 2338 | + | |
| 2339 | + | |
| 2340 | + | |
| 2341 | + | |
| 2342 | + | |
| 2343 | + | |
| 2344 | + | |
| 2345 | + | |
| 2346 | + | |
| 2347 | + | |
| 2348 | + | |
| 2349 | + | |
| 2350 | + | |
2300 | 2351 | | |
2301 | 2352 | | |
2302 | 2353 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
265 | 265 | | |
266 | 266 | | |
267 | 267 | | |
268 | | - | |
| 268 | + | |
| 269 | + | |
269 | 270 | | |
270 | 271 | | |
271 | 272 | | |
| |||
279 | 280 | | |
280 | 281 | | |
281 | 282 | | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
282 | 302 | | |
283 | 303 | | |
284 | 304 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
461 | 461 | | |
462 | 462 | | |
463 | 463 | | |
464 | | - | |
465 | | - | |
466 | | - | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
467 | 467 | | |
468 | | - | |
469 | | - | |
470 | | - | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
471 | 475 | | |
472 | 476 | | |
473 | 477 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
709 | 709 | | |
710 | 710 | | |
711 | 711 | | |
712 | | - | |
713 | | - | |
714 | | - | |
715 | | - | |
716 | | - | |
717 | | - | |
718 | | - | |
719 | | - | |
720 | | - | |
721 | | - | |
722 | | - | |
723 | | - | |
724 | | - | |
725 | | - | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
726 | 722 | | |
727 | 723 | | |
728 | 724 | | |
| |||
886 | 882 | | |
887 | 883 | | |
888 | 884 | | |
889 | | - | |
890 | | - | |
891 | | - | |
892 | | - | |
893 | | - | |
894 | | - | |
895 | | - | |
896 | | - | |
897 | | - | |
898 | | - | |
899 | 885 | | |
900 | | - | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
901 | 890 | | |
902 | | - | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
903 | 895 | | |
904 | 896 | | |
905 | 897 | | |
| |||
0 commit comments