@@ -468,22 +468,75 @@ Run `make` to update the user guide. The new configuration option will be added
468468
469469#### Update the Planner
470470
471- In ` native/core/src/execution/planner.rs ` , add a match case in the operator deserialization logic to handle your new protobuf message:
472-
473- ``` rust
474- use datafusion_comet_proto :: spark_operator :: operator :: OpStruct ;
475-
476- // In the create_plan or similar method:
477- match op . op_struct. as_ref () {
478- Some (OpStruct :: Scan (scan )) => {
479- // ... existing cases ...
480- }
481- Some (OpStruct :: YourNewOperator (your_op )) => {
482- create_your_operator_exec (your_op , children , session_ctx )
483- }
484- // ... other cases ...
485- }
486- ```
471+ Plan the operator in an ` OperatorBuilder ` , not in a match arm in ` PhysicalPlanner::create_plan ` .
472+ ` create_plan ` dispatches to ` OperatorRegistry ` first and only falls through to its own ` match ` for
473+ operators that have not moved yet, so new operators should not add code there.
474+
475+ 1 . Create a module under ` native/core/src/execution/planner/ ` , for example ` your_operator.rs ` , and
476+ declare it in ` native/core/src/execution/planner.rs ` . Implement ` OperatorBuilder ` for a unit
477+ struct. The builder plans the child operators through ` planner.create_plan ` and wraps the result
478+ in a ` SparkPlan ` :
479+
480+ ``` rust
481+ use std :: sync :: Arc ;
482+
483+ use datafusion :: physical_plan :: ExecutionPlan ;
484+ use datafusion_comet_proto :: spark_operator :: Operator ;
485+ use jni :: objects :: {Global , JObject };
486+
487+ use super :: {operator_registry :: OperatorBuilder , PhysicalPlanner , PlanCreationResult };
488+ use crate :: {execution :: spark_plan :: SparkPlan , extract_op};
489+
490+ /// Builder for `YourNewOperator`.
491+ pub struct YourNewOperatorBuilder ;
492+
493+ impl OperatorBuilder for YourNewOperatorBuilder {
494+ fn build (
495+ & self ,
496+ spark_plan : & Operator ,
497+ inputs : & mut Vec <Arc <Global <JObject <'static >>>>,
498+ partition_count : usize ,
499+ planner : & PhysicalPlanner ,
500+ ) -> PlanCreationResult {
501+ let your_op = extract_op! (spark_plan , YourNewOperator );
502+ let (scans , shuffle_scans , child ) =
503+ planner . create_plan (& spark_plan . children[0 ], inputs , partition_count )? ;
504+
505+ // Deserialize expressions and configuration, then create the execution plan.
506+ let exec = create_your_operator_exec (your_op , Arc :: clone (& child . native_plan))? ;
507+
508+ Ok ((
509+ scans ,
510+ shuffle_scans ,
511+ Arc :: new (SparkPlan :: new (spark_plan . plan_id, exec , vec! [child ])),
512+ ))
513+ }
514+ }
515+ ```
516+
517+ See ` explode.rs ` for a small builder and ` window.rs ` for one that rewrites its output.
518+
519+ 2 . In ` native/core/src/execution/planner/operator_registry.rs ` , add a variant to ` OperatorType ` ,
520+ map the ` OpStruct ` variant to it in ` get_operator_type ` , and add a ` register_*_operators ` method
521+ that inserts the builder, called from ` register_all_operators ` :
522+
523+ ``` rust
524+ OpStruct :: YourNewOperator (_ ) => Some (OperatorType :: YourNewOperator ),
525+ ```
526+
527+ ``` rust
528+ fn register_your_operator_operators (& mut self ) {
529+ use super :: your_operator :: YourNewOperatorBuilder ;
530+
531+ self . builders. insert (
532+ OperatorType :: YourNewOperator ,
533+ Box :: new (YourNewOperatorBuilder ),
534+ );
535+ }
536+ ```
537+
538+ Keep the helpers that only your operator uses in the same module, as ` aggregate.rs ` and
539+ ` join.rs ` do.
487540
488541` ProjectionBuilder ` prunes a DataFusion filter's output for column-only projections, including
489542empty projections such as the input to ` count(*) ` . Each required output column is filtered once,
@@ -493,16 +546,15 @@ plan ID. Computed projections and projections that use every input column are un
493546
494547#### Implement the Operator
495548
496- Create the operator implementation, either in an existing file or a new file in ` native/core/src/execution/operators/ ` :
549+ Create the operator implementation that the builder calls , either in an existing file or a new file in ` native/core/src/execution/operators/ ` :
497550
498551``` rust
499552use datafusion :: physical_plan :: {ExecutionPlan , ... };
500553use datafusion_comet_proto :: spark_operator :: YourNewOperator ;
501554
502555pub fn create_your_operator_exec (
503556 op : & YourNewOperator ,
504- children : Vec <Arc <dyn ExecutionPlan >>,
505- session_ctx : & SessionContext ,
557+ child : Arc <dyn ExecutionPlan >,
506558) -> Result <Arc <dyn ExecutionPlan >, ExecutionError > {
507559 // Deserialize expressions and configuration
508560 // Create and return the execution plan
0 commit comments