Skip to content

Commit af534e0

Browse files
authored
docs: document ReusedExchange caveat and both CometSparkToColumnar names in operator-count exclusions (#5240)
Follow-up to review feedback on #5206. - The transition-nodes bullet only listed `CometSparkRowToColumnar`, but `CometSparkToColumnarExec` also renders as `CometSparkColumnarToColumnar` when its child supports columnar. Both names are excluded from the operator counts today (the match is on the Scala type, not the rendered name), so list both. - `ReusedExchangeExec` is matched in the same ignore arm as the other wrapper nodes but was not mentioned. It also behaves differently from them: `getActualPlan` unwraps it to its child, so a reused subtree is counted once per reference rather than once per plan. Document it along with that caveat and the link to item 3 of #5203.
1 parent ee974a1 commit af534e0

1 file changed

Lines changed: 12 additions & 2 deletions

File tree

‎docs/source/user-guide/latest/understanding-comet-plans.md‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -182,12 +182,22 @@ Not every node in the plan is an eligible operator. The following are excluded
182182
from both operator counts:
183183

184184
- Transition nodes (`CometColumnarToRow`, `CometNativeColumnarToRow`,
185-
`CometSparkRowToColumnar`, `ColumnarToRow`, `RowToColumnar`), which are
186-
reported separately as the transition count.
185+
`CometSparkRowToColumnar`, `CometSparkColumnarToColumnar`, `ColumnarToRow`,
186+
`RowToColumnar`), which are reported separately as the transition count.
187+
`CometSparkRowToColumnar` and `CometSparkColumnarToColumnar` are the two names
188+
a single operator renders under, depending on whether its child already
189+
produces columnar data, and both are excluded.
187190
- Wrappers that do no work of their own: `AdaptiveSparkPlan`, `InputAdapter`,
188191
`WholeStageCodegen`, query stages, and `AQEShuffleRead`.
189192
- The reuse marker `ReusedSubquery`. The subquery it points at is counted where
190193
that subquery is shown, so the marker itself does not add to the totals.
194+
- `ReusedExchange`, but with a caveat: it is not cleanly excluded the way the
195+
wrappers above are. The node itself is skipped, and yet walking the plan
196+
replaces it with the exchange it reuses, so the reused subtree is counted once
197+
per reference rather than once for the whole plan. A plan that reuses one
198+
exchange in three places contributes that subtree's operators three times.
199+
Counting reused exchanges once is tracked as item 3 of
200+
[#5203](https://github.com/apache/datafusion-comet/issues/5203).
191201

192202
### `spark.comet.explain.native.enabled`
193203

0 commit comments

Comments
 (0)