Skip to content

Commit 9f81132

Browse files
committed
docs: note that the root duplicate field id check covers the whole data schema (#7006)
1 parent 515e51e commit 9f81132

1 file changed

Lines changed: 4 additions & 2 deletions

File tree

‎native/core/src/parquet/schema_adapter.rs‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -225,8 +225,10 @@ impl RootResolution {
225225

226226
/// Resolve every root file column against the requested root fields. Raises Spark's
227227
/// `_LEGACY_ERROR_TEMP_2094` "Found duplicate field(s)" when an id-bearing logical field would
228-
/// resolve to more than one physical field. That check covers every logical field, while an
229-
/// ambiguous name fails only for a referenced column (see `check_duplicate_columns`).
228+
/// resolve to more than one physical field. That check covers every logical field, which is the
229+
/// whole data schema rather than the projection, so it also fails a read that never references
230+
/// the field, where Spark reads the file (#7006). An ambiguous name fails only for a referenced
231+
/// column (see `check_duplicate_columns`).
230232
fn resolve_root_fields(resolver: &FieldResolver) -> DataFusionResult<Vec<RootResolution>> {
231233
let mut resolutions = vec![RootResolution::Unchanged; resolver.file_fields().len()];
232234
for requested in 0..resolver.requested_fields().len() {

0 commit comments

Comments
 (0)