You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit e618a80
Browse filesBrowse the repository at this point in the historyBrowse files
feat: extract enforcement into a PhysicalAnalyzerRule phase
Mirror the logical layer's AnalyzerRule/OptimizerRule split on the physical
side: an analyzer phase makes the plan valid (enforcing the distribution and
ordering requirements every operator declares) and the optimizer phase makes
the already-valid plan faster.
- Add a `PhysicalAnalyzerRule` trait and a `PhysicalAnalyzer` rule list,
plumbed through `Session` / `SessionState` / `SessionStateBuilder` the same
way physical optimizer rules are.
- `EnsureRequirements` now implements `PhysicalAnalyzerRule` and is the sole
default analyzer. It is kept as a `PhysicalOptimizerRule` too, so custom
rule lists that still register it keep working.
- The planner runs the analyzer phase at `EnsureRequirements`' historical
position (right before `CombinePartialFinalAggregate`), so the default
pipeline order and every plan are unchanged.
This is a prerequisite for the convergence-loop work in apache#25572: once
enforcement is its own phase, the optimizer rules can be re-run to a fixed
point without re-running the enforcement passes.
| 1 |`EnsureRequirements`| Enforces both distribution and sorting requirements in a single rule. |
76
+
67
77
### Physical Optimizer Rules
68
78
69
79
The same rule name may appear more than once when the default pipeline runs it
@@ -77,18 +87,17 @@ in multiple phases.
77
87
| 4 |`LimitedDistinctAggregation`| - | Pushes limit hints into grouped distinct-style aggregations when only a small result is needed. |
78
88
| 5 |`FilterPushdown`| pre-optimization phase | Pushes supported physical filters down toward data sources before distribution and sorting are enforced. |
79
89
| 6 |`WindowTopN`| - | Replaces eligible row-number window and filter patterns with per-partition TopK execution. |
80
-
| 7 |`EnsureRequirements`| - | Enforces both distribution and sorting requirements in a single idempotent rule. |
81
-
| 8 |`CombinePartialFinalAggregate`| - | Collapses adjacent partial and final aggregates when the distributed shape makes them redundant. |
82
-
| 9 |`OptimizeAggregateOrder`| - | Updates aggregate expressions to use the best ordering once sort requirements are known. |
83
-
| 10 |`ProjectionPushdown`| early pass | Pushes projections toward inputs before later physical rewrites add more limit and TopK structure. |
84
-
| 11 |`OutputRequirements`| remove phase | Removes the temporary output-requirement helper nodes after requirement-sensitive planning is done. |
85
-
| 12 |`LimitAggregation`| - | Passes a limit hint into eligible aggregations so they can keep fewer accumulator buckets. |
86
-
| 13 |`LimitPushPastWindows`| - | Pushes fetch limits through bounded window operators when doing so keeps the result correct. |
87
-
| 14 |`HashJoinBuffering`| - | Adds buffering on the probe side of hash joins so probing can start before build completion. |
88
-
| 15 |`LimitPushdown`| - | Moves physical limits into child operators or fetch-enabled variants to cut data early. |
89
-
| 16 |`TopKRepartition`| - | Pushes TopK below hash repartition when the partition key is a prefix of the sort key. |
90
-
| 17 |`ProjectionPushdown`| late pass | Runs projection pushdown again after limit and TopK rewrites expose new pruning opportunities. |
91
-
| 18 |`PushdownSort`| - | Pushes sort requirements into data sources that can already return sorted output. |
92
-
| 19 |`EnsureCooperative`| - | Wraps non-cooperative plan parts so long-running tasks yield fairly. |
93
-
| 20 |`FilterPushdown(Post)`| post-optimization phase | Pushes dynamic filters at the end of optimization, after plan references stop moving. |
94
-
| 21 |`SanityCheckPlan`| - | Validates that the final physical plan meets ordering, distribution, and infinite-input safety requirements. |
90
+
| 7 |`CombinePartialFinalAggregate`| - | Collapses adjacent partial and final aggregates when the distributed shape makes them redundant. |
91
+
| 8 |`OptimizeAggregateOrder`| - | Updates aggregate expressions to use the best ordering once sort requirements are known. |
92
+
| 9 |`ProjectionPushdown`| early pass | Pushes projections toward inputs before later physical rewrites add more limit and TopK structure. |
93
+
| 10 |`OutputRequirements`| remove phase | Removes the temporary output-requirement helper nodes after requirement-sensitive planning is done. |
94
+
| 11 |`LimitAggregation`| - | Passes a limit hint into eligible aggregations so they can keep fewer accumulator buckets. |
95
+
| 12 |`LimitPushPastWindows`| - | Pushes fetch limits through bounded window operators when doing so keeps the result correct. |
96
+
| 13 |`HashJoinBuffering`| - | Adds buffering on the probe side of hash joins so probing can start before build completion. |
97
+
| 14 |`LimitPushdown`| - | Moves physical limits into child operators or fetch-enabled variants to cut data early. |
98
+
| 15 |`TopKRepartition`| - | Pushes TopK below hash repartition when the partition key is a prefix of the sort key. |
99
+
| 16 |`ProjectionPushdown`| late pass | Runs projection pushdown again after limit and TopK rewrites expose new pruning opportunities. |
100
+
| 17 |`PushdownSort`| - | Pushes sort requirements into data sources that can already return sorted output. |
101
+
| 18 |`EnsureCooperative`| - | Wraps non-cooperative plan parts so long-running tasks yield fairly. |
102
+
| 19 |`FilterPushdown(Post)`| post-optimization phase | Pushes dynamic filters at the end of optimization, after plan references stop moving. |
103
+
| 20 |`SanityCheckPlan`| - | Validates that the final physical plan meets ordering, distribution, and infinite-input safety requirements. |
0 commit comments