Skip to content

Commit f6c69ce

Browse files
committed
test: cover a missing or negative deletion vector record count; drop the fixed day difference from the guide
1 parent 4eae287 commit f6c69ce

2 files changed

Lines changed: 21 additions & 6 deletions

File tree

‎docs/source/contributor-guide/iceberg-writes.md‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -246,9 +246,8 @@ Points where Comet adapts iceberg-rust to match iceberg-java:
246246
NULL past `chrono`'s calendar ([#6145](https://github.com/apache/datafusion-comet/issues/6145)).
247247
All four of its transforms floor a pre-1970 timestamp that lies exactly 999999 microseconds into a
248248
unit, which iceberg-java puts in the unit before
249-
([#6426](https://github.com/apache/datafusion-comet/issues/6426)), and its `day` puts some
250-
timestamps from the last second before a pre-1970 midnight in the next day. Every other transform
251-
stays on iceberg-rust.
249+
([#6426](https://github.com/apache/datafusion-comet/issues/6426)). Every other transform stays on
250+
iceberg-rust.
252251
- **File names.** `file_name_prefix` embeds the partition id, the task attempt id and the operation
253252
id, so a retried or speculative attempt never reuses another attempt's file names.
254253
- **Row pacing.** iceberg-java's rolling writer checks the target file size every 1000 rows of the

‎native/core/src/execution/planner.rs‎

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8216,10 +8216,12 @@ mod tests {
82168216
assert_eq!(fields[1].name, "category");
82178217
}
82188218

8219-
/// iceberg-rust holds a deletion vector's blob coordinates as `u64`, so the planner rejects a
8220-
/// negative one from the proto rather than build the delete file with it.
8219+
/// iceberg-rust holds a deletion vector's record count and blob coordinates as `u64`, so the
8220+
/// planner rejects a negative one from the proto rather than build the delete file with it. A
8221+
/// deletion vector without a record count reaches iceberg-rust's builder, whose error must
8222+
/// come back through the `Invalid Iceberg delete file` mapping.
82218223
#[test]
8222-
fn test_delete_file_pool_rejects_negative_coordinates() {
8224+
fn test_delete_file_pool_rejects_invalid_deletion_vectors() {
82238225
let deletion_vector = spark_operator::IcebergDeleteFile {
82248226
file_path_idx: 0,
82258227
content_type: "POSITION_DELETES".to_string(),
@@ -8255,6 +8257,20 @@ mod tests {
82558257
},
82568258
"negative content size",
82578259
),
8260+
(
8261+
spark_operator::IcebergDeleteFile {
8262+
record_count: Some(-1),
8263+
..deletion_vector.clone()
8264+
},
8265+
"negative record count",
8266+
),
8267+
(
8268+
spark_operator::IcebergDeleteFile {
8269+
record_count: None,
8270+
..deletion_vector.clone()
8271+
},
8272+
"is missing record_count",
8273+
),
82588274
] {
82598275
let error = parse(delete).expect_err(expected).to_string();
82608276
assert!(

0 commit comments

Comments
 (0)