Repository navigation
Commit ba2c820
committed
feat: report the ordering and partitioning of materialized CTE scans
A `MaterializedCteScanExec` reported no ordering and unknown partitioning, so the continuation sorted or repartitioned rows that the CTE body already sorted or partitioned.
`optimize_physical_plan` now optimizes each CTE body on its own first, as the child of a `MaterializedCteExec` with an empty continuation.
A body that reads another CTE is optimized after that CTE.
Each scan then gets the partition count, the output ordering and the hash partitioning of its optimized body, and the full plan is optimized one time.
The result is used only if each body in it still has the properties that its scans report.
If not, the plan with statistics only is optimized and used.
A scan with the properties of the body replays one body partition per partition, and returns an internal error if the partition counts are not the same at execution.
Closes apache#26424.1 parent 19fa23e commit ba2c820
4 files changed
Lines changed: 1025 additions & 62 deletions
File tree
- datafusion
- core/src
- datasource
- physical-plan/src
- sqllogictest/test_files
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
527 | 527 | | |
528 | 528 | | |
529 | 529 | | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
530 | 584 | | |
0 commit comments