You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
//! List-aware predicates (for example, `array_has`, `array_has_all`, and
//! `array_has_any`) can be evaluated directly during Parquet decoding. Struct
//! columns and other nested projections that are not explicitly supported will
//! continue to be evaluated after the batches are materialized.
Projections read the entire column instead of the single leaf column (no special handling). We essentially need to do something more sophisticated when going from projection PhysicalExpr -> Parquet leaf indexes. We currently collect all columns and map to root indexes. We need to detect get_field(col, 'f') and map that to a Parquet leaf index.
I tried to fix these in ParquetOpener and found that we'd get errors from arrow-rs because of apache/arrow-rs#9066.
Activity
Allow filters on struct fields to be pushed down into Parquet scan (#…
Project only accessed struct leaves in Parquet row filter pushdown (#…
Allow filters on struct fields to be pushed down into Parquet scan (a…
Project only accessed struct leaves in Parquet row filter pushdown (a…