Support Iceberg 1.12.0 and run the Iceberg suites on Spark 4.2 #7493

What is the problem the feature request solves?

Iceberg 1.12.0 (released 2026-09-30) is the first Iceberg release with a Spark 4.2 module. Comet does not use it yet:

• The spark-4.2 profile in spark/pom.xml reuses iceberg-spark-runtime-4.0_2.13:1.10.0, because no 4.2 runtime existed. That jar is binary-incompatible with Spark 4.2 (connector.catalog.View became a class), so icebergAvailable is false on 4.2 and CometIcebergTestBase and CometFuzzIcebergBase skip every Iceberg suite there. The native Iceberg REST catalog scan test is also skipped (Spark 4.2: native Iceberg REST catalog scan test fails under Comet #4969).
• No CI profile covers Iceberg 1.12. There is no dev/diffs/iceberg/1.12.0.diff, and Iceberg's own suites do not run on Spark 4.2 at all. The existing jobs are iceberg_1_8 through iceberg_1_11, and the newest pairs with Spark 4.1.3.
• Iceberg 1.12 removed the public 2-arg ManifestFiles.read(ManifestFile, FileIO), so native Iceberg writes silently fall back (Native Iceberg write silently falls back on Iceberg 1.12: ManifestFiles.read(ManifestFile, FileIO) was removed #7240). A profile on 1.12 would have caught this.

Related: #4871, #4969, #7240, #5649.

Describe the potential solution

One PR, one commit per part:

1. Point the spark-4.2 profile at iceberg-spark-runtime-4.2_2.13:1.12.0. Drop the !isSpark42Plus guards in the Iceberg test bases and re-enable the REST catalog scan test (Spark 4.2: native Iceberg REST catalog scan test fails under Comet #4969) if it passes.
2. Fix Native Iceberg write silently falls back on Iceberg 1.12: ManifestFiles.read(ManifestFile, FileIO) was removed #7240 by resolving the 3-arg ManifestFiles.read(manifest, io, specsById), which is public on every Iceberg version Comet supports (1.5.2 through 1.12).
3. Add dev/diffs/iceberg/1.12.0.diff, ported from the 1.11.0 diff and regenerated against the apache-iceberg-1.12.0 tag, plus an iceberg_1_12 CI job (Iceberg 1.12.0 on Spark 4.2.0, nightly tier and the run-iceberg-tests label). That touches ci.yml, dev/ci/compute-changes.py, dev/ci/check-ci-config.py and the workflows README.
4. Update iceberg-spark-tests.md, iceberg-writes.md (which says the Iceberg suites skip on 4.2) and the Iceberg compatibility notes.

Open question for #4871: whether 1.12 should join the queue tier or stay nightly. Spark 4.2 is still experimental, so nightly plus the label seems right for now.

Additional context

I checked the release tags in apache/iceberg: knownSparkVersions is 3.4,3.5,4.0 on 1.10.2, 3.4,3.5,4.0,4.1 on 1.11.0, and 3.5,4.0,4.1,4.2 on 1.12.0 (Spark 4.2.0). I could not confirm the iceberg-spark-runtime-4.2_2.13:1.12.0 artifact on Maven Central, so the first step is to check that it resolves.