Repository navigation
Commit e8e3670
feat(parquet): clip nested wrappers during schema pruning (apache#24315)
## Which issue does this PR close?
- Closes apache#24122.
## Rationale for this change
Parquet nested schema pruning currently descends through `Struct`,
`List`, and `LargeList`. Narrowing casts through `ListView`,
`LargeListView`, `FixedSizeList`, or `Dictionary` therefore read
physical leaves that the cast never consumes.
The issue also listed `RunEndEncoded`, but its casts do not use
DataFusion's name-based nested cast path. Arrow may map nested struct
fields positionally, so clipping those values could change results; they
must remain a full read.
## What changes are included in this PR?
Extend nested schema pruning through matching `ListView`,
`LargeListView`, `FixedSizeList`, and `Dictionary` wrappers while
preserving safe full-read fallbacks.
## Are these changes tested?
Yes, with new and existing tests.
## Are there any user-facing changes?
No.1 parent f660ee5 commit e8e3670
1 file changed
Lines changed: 390 additions & 92 deletions
0 commit comments