You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Adopt a pin-bump policy for the iceberg-rust dependency #5645
native/Cargo.toml pins iceberg and iceberg-storage-opendal to a git revision (3d84c81353b1b23b6e4ae8eea8f8a021cc6927a7) rather than a release. The pin is what the native Iceberg scan and writer are tested against, but nothing decides when it moves, and #5636 is the first bug caused by it drifting behind an upstream fix (partition-path escaping, apache/iceberg-rust#2875). With the writer now producing files that iceberg-java reads back, a stale pin can turn an upstream bug fix into a Comet correctness bug.
Describe the potential solution
Write down and follow a policy, for example:
Bump the pin at least once per Comet release cycle, and immediately when an upstream fix affects file bytes, manifest metadata, or partition layout.
Prefer tagged iceberg-rust releases over arbitrary revisions once a release contains everything Comet needs, so the version is visible in Cargo.lock and in the docs.
Each bump runs the Iceberg suites on all four Spark/Iceberg profiles plus the manifest-metrics parity tests, since those are what catch behavior changes in the writer.
Track "waiting on upstream" items (see the eligibility-restrictions issue) so a bump can also lift restrictions.
Note the bump in the release notes, since users reading Comet-written tables with iceberg-java care which writer semantics they got.
Additional context
Part of the native Iceberg writes epic, #5649. Related: #5636.
Activity