Skip to content

Commit 60af6cb

Browse files
docs: match the shuffle read condition, drop native and seamless from the home page
The Shuffle Reads paragraph in plugin_overview.md now gives the same ShuffleScan condition as the paragraph above it and jvm_shuffle.md. The overview figure and the index.md compatibility blurb write Rust instead of native (N3) and drop seamlessly (P10). Generated-by: OpenCode
1 parent 491f561 commit 60af6cb

3 files changed

Lines changed: 12 additions & 11 deletions

File tree

‎docs/source/_static/images/comet-overview.svg‎

Lines changed: 6 additions & 6 deletions
Loading

‎docs/source/contributor-guide/plugin_overview.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -199,8 +199,9 @@ IPC. See [JVM Shuffle](jvm_shuffle.md) for details.
199199

200200
For shuffle reads, `CometShuffledBatchRDD` requests a `ShuffleReader` from the shuffle manager and gets a
201201
`CometBlockStoreShuffleReader`, which fetches blocks with Spark's `ShuffleBlockFetcherIterator`. Both shuffle
202-
implementations write the same Arrow IPC block format, so the same reader serves both. When a native plan consumes the
203-
shuffle output and `spark.comet.shuffle.directRead.enabled` is `true`, the default, the compressed blocks are passed
204-
to that plan, which decodes them itself. Otherwise `NativeBatchDecoderIterator` decodes each block in native
205-
code through JNI, and Arrow FFI imports the result into the JVM as a `ColumnarBatch`. See
202+
implementations write the same Arrow IPC block format, so the same reader serves both. When a Comet block reads the
203+
shuffle output as a `ShuffleScan` input (see [Query Execution](#query-execution)) and
204+
`spark.comet.shuffle.directRead.enabled` is `true`, the default, the compressed IPC blocks go to its Rust plan,
205+
which decodes them. Otherwise `NativeBatchDecoderIterator` decodes each block in native code through JNI, and Arrow
206+
FFI imports the result into the JVM as a `ColumnarBatch`. See
206207
[Read Path](native_shuffle.md#read-path) for details.

‎docs/source/index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ Speeds up your existing Spark queries by running them on Apache Arrow columnar d
7878
<div class="comet-feature__copy">
7979
<p class="comet-feature__eyebrow">Spark Compatibility</p>
8080
<h2 class="comet-feature__title">100% compatibility with supported Spark versions.</h2>
81-
<p class="comet-feature__body">Comet aims for 100% compatibility with all supported versions of Apache Spark, allowing you to integrate Comet into your existing Spark deployments and workflows seamlessly. With no code changes required, you can immediately harness the benefits of Comet's acceleration capabilities without disrupting your Spark applications. The Comet extension automatically detects unsupported features and falls back to the Spark engine.</p>
81+
<p class="comet-feature__body">Comet aims for 100% compatibility with all supported versions of Apache Spark, so you can add Comet to existing Spark deployments and workflows without changing any code. The Comet extension automatically detects unsupported features and falls back to the Spark engine.</p>
8282
<p class="comet-feature__links">
8383
<a href="user-guide/latest/compatibility/spark-versions.html">Spark version compatibility &rarr;</a>
8484
</p>

0 commit comments

Comments
 (0)