Skip to content

Commit bdd2aeb

Browse files
authored
docs: correct Spark 4.2 version and CI test status in installation guide (#5315)
The installation guide listed the experimental Spark 4.2 support as targeting `4.2.0-preview4` and claimed no Comet tests run in CI. Both are stale: - The `spark-4.2` Maven profile targets the `4.2.0` final release. - `pr_build_linux.yml` includes a "Spark 4.2, JDK 17" job in the `linux-test` matrix, so Comet tests do run in CI. Spark SQL tests still do not run for 4.2 (no `spark_4_2` job in `ci.yml`), so that column stays "No" and the version remains experimental. Also refresh the two `4.2.0-preview4` examples in the contributor guide, which cited a version Comet no longer targets, to use a hypothetical future release instead.
1 parent c706360 commit bdd2aeb

3 files changed

Lines changed: 9 additions & 8 deletions

File tree

‎docs/source/contributor-guide/adding_a_new_spark_version.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,8 @@ existing profile (for example `spark-4.1`) and update the version
6262
properties:
6363

6464
- `spark.version`: the full upstream version, including any qualifier
65-
(for example `4.2.0-preview4`).
65+
(for example `4.3.0` for a final release, or `4.3.0-preview1` while only a
66+
preview has been published).
6667
- `spark.version.short`: the major.minor (for example `4.2`).
6768
- `parquet.version`, `slf4j.version`, `scala.version`,
6869
`scala.binary.version`, `java.version`: align with what the new Spark
@@ -317,7 +318,7 @@ issue keeps the diff small and makes regressions easy to bisect.
317318
The user guide currently uses two tiers, "Supported" and "Experimental".
318319
Comet uses "Experimental" to describe its confidence in its own
319320
integration with a Spark version, distinct from Spark's "preview" tag
320-
(which refers to upstream release qualifiers like `4.2.0-preview4`). The
321+
(which refers to upstream release qualifiers like `4.3.0-preview1`). The
321322
term is already established in `installation.md`, `operators.md`, and
322323
`datasources.md`, so keep using it rather than introducing a new label.
323324

‎docs/source/user-guide/latest/compatibility/spark-versions.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,9 +91,9 @@ Spark 4.1.3 is supported with Java 17/21 and Scala 2.13.
9191

9292
## Spark 4.2 (Experimental)
9393

94-
Spark 4.2.0-preview4 is provided as experimental support with Java 17 and Scala 2.13.
94+
Spark 4.2.0 is provided as experimental support with Java 17 and Scala 2.13.
9595

9696
```{warning}
97-
Spark 4.2 support is experimental and targets a preview release of Spark. It is intended for early
98-
evaluation only and should not be used in production.
97+
Spark 4.2 support is experimental. Comet tests run in CI for this version, but the Spark SQL tests
98+
do not yet. It is intended for early evaluation only and should not be used in production.
9999
```

‎docs/source/user-guide/latest/installation.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -62,9 +62,9 @@ Note that we do not test the full matrix of supported Java and Scala versions in
6262
Experimental support is provided for the following versions of Apache Spark and is intended for development/testing
6363
use only and should not be used in production yet.
6464

65-
| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark SQL Tests in CI |
66-
| -------------- | ------------ | ------------- | ----------------- | --------------------- |
67-
| 4.2.0-preview4 | 17 | 2.13 | No | No |
65+
| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark SQL Tests in CI |
66+
| ------------- | ------------ | ------------- | ----------------- | --------------------- |
67+
| 4.2.0 | 17 | 2.13 | Yes | No |
6868

6969
Note that Comet may not fully work with proprietary forks of Apache Spark such as the Spark versions offered by
7070
Cloud Service Providers.

0 commit comments

Comments
 (0)