filter_row_groups_by_byte_range duplicates rows for sub-row-group file splits #2614

Apache Iceberg Rust version

Latest (main), also reproduces on rev 83b4595.

Describe the bug

ArrowReader::filter_row_groups_by_byte_range (added in #1779) selects a row group for every FileScanTask byte range that overlaps it:

// crates/iceberg/src/arrow/reader/row_filter.rs
if current_byte_offset < end && start < row_group_end {
    selected.push(idx);
}

When a data file is split into byte ranges smaller than a row group, multiple tasks overlap the same row group, so each one reads it and its rows are duplicated.

iceberg-rust's own planner never produces such splits: it emits a single whole-file task per file (start=0, length=file_size, see ManifestEntryContext::into_file_scan_task), so this stays latent for internal use. But ArrowReader::read is a public entry point, and external engines that plan with Iceberg-Java/Spark and drive ArrowReader directly do hit it. Apache DataFusion Comet is the concrete case (apache/datafusion-comet#4590): Spark tiles a file into split-size chunks regardless of row-group layout, producing many sub-row-group byte ranges, and the native Iceberg scan returns duplicate rows.

parquet-java assigns each row group to exactly one split by midpoint ownership: a split owns a row group only if its range contains the row group's midpoint (ParquetMetadataConverter.filterFileMetaDataByMidpoint):

long midPoint = startIndex + totalSize / 2;
if (filter.contains(midPoint)) {
  newRowGroups.add(rowGroup);
}

filter_row_groups_by_byte_range should do the same. Because splits tile the file contiguously and disjointly, exactly one split's range contains a given midpoint, so each row group is read once. This is a no-op for whole-file tasks (every midpoint lies in [0, file_size)), so iceberg-rust's own scan path is unchanged.

This was partially addressed by #1779, which added byte-range filtering but used overlap rather than midpoint semantics. The overlap formula happens to be correct only when splits land exactly on row-group boundaries (which the existing test_file_splits_respect_byte_ranges covers), so the sub-row-group case slipped through.

To Reproduce

Write a Parquet file with 3 row groups of 100 rows each, tile the whole file into 64-byte splits, read each split, and count rows. A 64-byte split is far smaller than a row group, so the overlap formula reads each row group from several splits and returns roughly 2800 rows instead of 300. Whole-file reads (start=0, length=0 or length=file_size) are unaffected.

Expected behavior

Each row group is read by exactly one split, so the total rows across all splits equals the file's row count, matching parquet-java / Iceberg-Java semantics.

Willingness to contribute

I can contribute the fix independently (patch and regression test ready).