Skip to content

Commit 7caaa3c

Browse files
committed
build: default to a Paimon 2.0 bundle — deletion vectors become the norm
The binary-primary-key lookup comparator fix (apache/paimon#8873) merged upstream, and Apache's published 2.0 snapshots now carry it — the first Paimon the deletion-vector state tables can actually run on. Pinning the snapshot as the default bundle (resolved from the Apache snapshots repository; swap in a release the moment one ships with the fix) makes deletion vectors the mode every maintained deployment gets: barrier-time record-level TTL expiry, raw-parquet reads, and no merge-read reclamation lag. The capability probe stays — an older bundle still degrades to merge-read tables instead of failing at the first post-restore barrier — but it is now the exception path, not the default. Bumping the bundle also un-skips the whole deletion-vector test suite in a default build, which surfaced one casualty: the end-to-end ServiceLoader maintenance test predates batch coalescing and had been skipped ever since the 1.4.2 default made its probe fail, so it never received the coalescing-off harness extension its sibling suites got. Verified stale (the pre-TTL commit fails it identically on this bundle) and fixed.
1 parent b622a8b commit 7caaa3c

3 files changed

Lines changed: 22 additions & 5 deletions

File tree

‎pom.xml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -356,6 +356,15 @@
356356
<id>confluent</id>
357357
<url>https://packages.confluent.io/maven/</url>
358358
</repository>
359+
<!-- Hosts the timestamped Paimon 2.0 snapshot bundle the state compactor pins (the first
360+
Paimon carrying the binary-primary-key lookup comparator fix, which deletion-vector
361+
state tables require); drop once a Paimon release includes it. -->
362+
<repository>
363+
<id>apache-snapshots</id>
364+
<url>https://repository.apache.org/content/groups/snapshots/</url>
365+
<releases><enabled>false</enabled></releases>
366+
<snapshots><enabled>true</enabled></snapshots>
367+
</repository>
359368
</repositories>
360369

361370
<build>

‎streamfusion-paimon-compactor/pom.xml‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,13 @@
1313
<artifactId>streamfusion-paimon-compactor</artifactId>
1414

1515
<properties>
16-
<paimon.bundle.version>1.4.2</paimon.bundle.version>
16+
<!-- The Apache 2.0 snapshot (snapshots published after 2026-07-29 carry the
17+
binary-primary-key lookup comparator fix, apache/paimon#8873) — the first Paimon
18+
deletion-vector state tables can run on, and deletion vectors are the canonical
19+
production mode for native state tables. Pin a stable release here the moment one
20+
ships with the fix. Overridable for older bundles, where the capability probe falls
21+
the backend back to merge-read tables. -->
22+
<paimon.bundle.version>2.0-SNAPSHOT</paimon.bundle.version>
1723
</properties>
1824

1925
<!-- Java Paimon as the maintainer of native state tables: stock Paimon's compaction (its own
@@ -34,10 +40,9 @@
3440
<dependency>
3541
<groupId>org.apache.paimon</groupId>
3642
<artifactId>paimon-bundle</artifactId>
37-
<!-- Overridable so the deletion-vector suites can run against a Paimon carrying the
38-
binary-primary-key lookup comparator fix before it reaches a release
39-
(-Dpaimon.bundle.version=2.0-SNAPSHOT with a locally built bundle). On releases
40-
without the fix the deletion-vector tests skip via the capability probe. -->
43+
<!-- Overridable (-Dpaimon.bundle.version=...). On bundles without the comparator
44+
fix the deletion-vector tests skip via the capability probe and the backend
45+
runs merge-read tables. -->
4146
<version>${paimon.bundle.version}</version>
4247
<scope>provided</scope>
4348
</dependency>

‎streamfusion-paimon-compactor/src/test/java/io/github/jordepic/streamfusion/paimon/JavaPaimonStateCompactorTest.java‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,10 @@
3333
import org.apache.paimon.table.FileStoreTableFactory;
3434
import org.apache.paimon.table.source.DataSplit;
3535
import org.apache.paimon.table.source.Split;
36+
import io.github.jordepic.streamfusion.operator.CoalescingOff;
3637
import org.junit.jupiter.api.Assumptions;
3738
import org.junit.jupiter.api.BeforeAll;
39+
import org.junit.jupiter.api.extension.ExtendWith;
3840
import org.junit.jupiter.api.Test;
3941

4042
/**
@@ -44,6 +46,7 @@
4446
* lookup-wait model) — visible as a level-0-free table with a deletion-vector index and an
4547
* unchanged changelog.
4648
*/
49+
@ExtendWith(CoalescingOff.class)
4750
class JavaPaimonStateCompactorTest {
4851

4952
@BeforeAll

0 commit comments

Comments
 (0)