You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 5550f42
Browse filesBrowse the repository at this point in the historyBrowse files
fix: preserve array field dispatch and reuse metadata
Keep field lookup within list dispatch so invalid children retain the
intended error, and reuse field metadata unless nullability must widen.
Simplify the complex-null guard and document why post-scan filters stay.
Move the required-field regression into the native Iceberg suite, cover
null lists and SQL null elements, and retain explicit expected-row checks.
Copy file name to clipboardExpand all lines: docs/source/user-guide/latest/iceberg.md
+9-7Lines changed: 9 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -98,13 +98,6 @@ The native Iceberg reader supports the following features:
98
98
-`IN` and `NOT IN` list operations
99
99
-`BETWEEN` operations
100
100
101
-
Native scanning does not imply that every predicate is evaluated inside iceberg-rust.
102
-
List, map, and struct NULL checks can use native scans. Their residuals are omitted
103
-
before native planning; the retained post-scan filter enforces them. Empty collections
104
-
and collections containing null elements are non-null, matching Spark. A conjunction
105
-
containing one of these residuals currently loses native row-group pruning for primitive conjuncts;
106
-
safe partial pruning is tracked in [#5883](https://github.com/apache/datafusion-comet/issues/5883).
107
-
108
101
**Partitioning:**
109
102
110
103
- Standard partitioning with partition pruning
@@ -119,6 +112,15 @@ safe partial pruning is tracked in [#5883](https://github.com/apache/datafusion-
119
112
- Hadoop Distributed File System (HDFS)
120
113
- S3-compatible storage (AWS S3, MinIO)
121
114
115
+
### Predicate pushdown vs native scanning
116
+
117
+
Native scanning does not imply that every predicate is evaluated inside iceberg-rust.
118
+
List, map, and struct NULL checks can use native scans. Their residuals are omitted
119
+
at serialization time; the retained post-scan filter enforces them. Empty collections
120
+
and collections containing null elements are non-null, matching Spark. A conjunction
121
+
containing one of these residuals currently loses native row-group pruning for primitive conjuncts;
122
+
safe partial pruning is tracked in [#5883](https://github.com/apache/datafusion-comet/issues/5883).
123
+
122
124
### REST Catalog
123
125
124
126
Comet's native Iceberg reader also supports REST catalogs. The following example shows how to
0 commit comments