Skip to content

Commit bc3ffc2

Browse files
docs: say what decides the CometCollectLimit path, fix two overview figure labels
CometCollectLimit takes rows on the JVM for collect, show and take, through executeCollect, and runs Rust limit plans otherwise; where it sits in the plan does not decide this. The overview figure no longer says every supported operator runs in Rust (CometUnion does not), and drops vectorized as a synonym for columnar (T10). Generated-by: OpenCode
1 parent 60af6cb commit bc3ffc2

2 files changed

Lines changed: 13 additions & 13 deletions

File tree

‎docs/source/_static/images/comet-overview.svg‎

Lines changed: 3 additions & 3 deletions
Loading

‎docs/source/user-guide/latest/understanding-comet-plans.md‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -342,16 +342,16 @@ through a single-partition Comet shuffle.
342342
These do their own work on the JVM and participate in the Comet pipeline. Some of them also run
343343
Rust plans, as their rows say.
344344

345-
| Node | Notes |
346-
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- |
347-
| `CometUnion` | JVM-side union of Comet inputs. The Rust side reads each branch as a separate scan. |
348-
| `CometCoalesce` | JVM-side partition coalesce. |
349-
| `CometCollectLimit` | Collect limit (`CollectLimitExec`). On the JVM when Spark collects it at the plan root; otherwise it runs Rust limit plans. |
350-
| `CometBroadcastExchange` | Broadcast exchange producing serialized Arrow batches that the consumer can decode. |
351-
| `CometSubqueryBroadcast` | Companion to `CometBroadcastExchange` for dynamic partition pruning subqueries. |
352-
| `CometMapInBatch` | Runs `mapInArrow` / `mapInPandas` Python UDFs on Comet's Arrow batches (experimental). See [PyArrow UDF Acceleration](pyarrow-udfs.md). |
353-
| `IcebergWrite` | Executor-side Iceberg data-file write in Comet's split-operator Iceberg write plan. See [Iceberg Writes](iceberg.md#writes). |
354-
| `IcebergCommit` | Driver-side commit for Comet's split-operator Iceberg write plan. |
345+
| Node | Notes |
346+
| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------- |
347+
| `CometUnion` | JVM-side union of Comet inputs. The Rust side reads each branch as a separate scan. |
348+
| `CometCoalesce` | JVM-side partition coalesce. |
349+
| `CometCollectLimit` | Collect limit (`CollectLimitExec`). `collect`, `show`, and `take` take rows from its child on the JVM; other reads run Rust limit plans. |
350+
| `CometBroadcastExchange` | Broadcast exchange producing serialized Arrow batches that the consumer can decode. |
351+
| `CometSubqueryBroadcast` | Companion to `CometBroadcastExchange` for dynamic partition pruning subqueries. |
352+
| `CometMapInBatch` | Runs `mapInArrow` / `mapInPandas` Python UDFs on Comet's Arrow batches (experimental). See [PyArrow UDF Acceleration](pyarrow-udfs.md). |
353+
| `IcebergWrite` | Executor-side Iceberg data-file write in Comet's split-operator Iceberg write plan. See [Iceberg Writes](iceberg.md#writes). |
354+
| `IcebergCommit` | Driver-side commit for Comet's split-operator Iceberg write plan. |
355355

356356
### Shuffle Operators
357357

0 commit comments

Comments
 (0)