@@ -1352,34 +1352,65 @@ analyzer list is the former optimizer list up to and including
13521352` EnsureRequirements ` (` OutputRequirements ` add mode, ` aggregate_statistics ` ,
13531353` EliminateConstantJoin ` , ` join_selection ` , ` LimitedDistinctAggregation ` ,
13541354` FilterPushdown ` , ` WindowTopN ` , ` EnsureRequirements ` ), in the same order, so
1355- default plans do not change. The
1356- ` PhysicalOptimizerRule ` s then run on that plan. ` EXPLAIN VERBOSE ` shows the
1357- analyzer passes as well.
1358-
1359- ** Migration guide:**
1360-
1361- - A type that implements the ` Session ` trait and overrides
1355+ default plans do not change. ` EXPLAIN VERBOSE ` and the observer passed to
1356+ ` DefaultPhysicalPlanner::optimize_physical_plan ` show the analyzer rules as
1357+ well.
1358+
1359+ ** Who is affected and what to do:**
1360+
1361+ - ** Rules placed in front of the default optimizer list.** A list built as
1362+ your rules followed by ` PhysicalOptimizer::new().rules ` used to run your
1363+ rules before ` EnsureRequirements ` . They now run after the whole analyzer
1364+ phase, enforcement included, and nothing enforces the plan again after them.
1365+ A rule that has to run before enforcement belongs in the analyzer list, in
1366+ front of the default rules and wrapped in ` RunInAnalyzer ` :
1367+
1368+ ``` rust,ignore
1369+ use datafusion::physical_optimizer::analyzer::{PhysicalAnalyzer, RunInAnalyzer};
1370+
1371+ let mut analyzer_rules: Vec<Arc<dyn PhysicalAnalyzerRule + Send + Sync>> =
1372+ vec![Arc::new(RunInAnalyzer::new(Arc::new(MyRule::new())))];
1373+ analyzer_rules.extend(PhysicalAnalyzer::new().rules);
1374+ let state = SessionStateBuilder::new()
1375+ .with_default_features()
1376+ .with_physical_analyzer_rules(analyzer_rules)
1377+ .build();
1378+ ```
1379+
1380+ A rule added with ` with_physical_analyzer_rule ` runs after
1381+ ` EnsureRequirements ` , so it must leave the plan valid.
1382+
1383+ - ** An empty optimizer list no longer skips enforcement.**
1384+ ` with_physical_optimizer_rules(vec![]) ` still runs the analyzer phase, so the
1385+ plan has the repartitions and sorts enforcement adds. To plan without them
1386+ as before, also call ` with_physical_analyzer_rules(vec![]) ` .
1387+
1388+ - ** Code that wraps, lists or runs the rules itself.**
1389+ ` SessionState::physical_optimizers() ` no longer contains the analyzer rules.
1390+ Tools that wrap every rule, for tracing or for ` EXPLAIN ` output, or that
1391+ list rule names, should handle ` physical_analyzers() ` too. Code that runs the
1392+ rules itself should call ` DefaultPhysicalPlanner::optimize_physical_plan ` , or
1393+ run the ` physical_analyzers() ` rules first.
1394+
1395+ - ** Custom optimizer lists that still contain analyzer rules.** A list copied
1396+ from the old default runs ` join_selection ` , ` EnsureRequirements ` and the
1397+ other analyzer rules twice. Results stay correct, but planning takes longer
1398+ and plans can change. Drop those rules from the list.
1399+
1400+ - ** Custom ` Session ` implementations.** A type that overrides
13621401 ` physical_optimizers ` should also override ` physical_analyzers ` , for example
1363- by returning ` SessionState::physical_analyzers ` . The default implementation
1364- returns no rules, so such a session would otherwise run no enforcement.
1365- - The default analyzer rules run even when a custom optimizer list is set with
1366- ` SessionStateBuilder::with_physical_optimizer_rules ` . A custom list that still
1367- contains a rule now in the analyzer (for example ` EnsureRequirements ` ) runs it
1368- twice; drop it from the list, or pass ` with_physical_analyzer_rules(vec![]) `
1369- to keep the previous single-list behavior.
1370- - Code that runs the ` physical_optimizers() ` rules itself, instead of calling
1371- ` DefaultPhysicalPlanner::optimize_physical_plan ` , must run the
1372- ` physical_analyzers() ` rules first, the same way. Without them the plan has
1373- no enforcement and no record of the query's output ordering.
1374- - To run a built-in ` PhysicalOptimizerRule ` in a custom analyzer list, wrap it
1375- in ` RunInAnalyzer ` .
1376- - In debug builds, planning fails where the analyzer phase ends if the plan is
1377- not executable or leaves a distribution or ordering requirement unmet. A
1378- custom analyzer list must therefore end with ` EnsureRequirements ` , or repair
1379- the plan with
1402+ by returning ` SessionState::physical_analyzers ` . The default returns no
1403+ rules, so the session plans without enforcement and fails at planning or at
1404+ execution.
1405+
1406+ - ** Debug builds** check where the analyzer phase ends that the plan is
1407+ executable and meets every distribution and ordering requirement. A custom
1408+ analyzer list must end with ` EnsureRequirements ` , or repair the plan with
13801409 ` datafusion_physical_optimizer::ensure_requirements::satisfy_requirements ` ,
13811410 which inserts only the missing repartitions and sorts. A session with no
13821411 analyzer rules is not checked.
1383- - The observer passed to ` DefaultPhysicalPlanner::optimize_physical_plan ` now
1384- receives the rule name: ` FnMut(&dyn ExecutionPlan, &str) ` instead of
1385- ` FnMut(&dyn ExecutionPlan, &dyn PhysicalOptimizerRule) ` .
1412+
1413+ - ** ` EnsureRequirements ` on a plan that already has repartitions and sorts**
1414+ now removes them first and places them again. Nothing changes in the default
1415+ pipeline, but code that runs it more than once, or on a plan whose
1416+ enforcement is already in place, can get different plans.
0 commit comments