Skip to content

Implement MultiStage PREWHERE for ParquetReaderV3 #89101

@alesapin

Description

@alesapin

Right now PREWHERE in ParquetReaderV3 is implemented in a single step (actually two steps, one of them is row-level security): link. This simplifies implementation, but not that effective, because during query analysis ClickHouse push ALL conditions to PREWHERE: setting + usage.

The idea is to implement Multistage PREWHERE similar to what MergeTree has.

Workaround solution is to disable move_all_conditions_to_prewhere for parquet and move only best condition to PREWHERE.

Metadata

Metadata

Assignees

Labels

comp-datalakeData lake table formats (Iceberg/Delta/Hudi) integration.comp-formatsInput/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).performance

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions