-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Cache Parquet footer in LRU Cache #89102
Copy link
Copy link
Labels
comp-datalakeData lake table formats (Iceberg/Delta/Hudi) integration.Data lake table formats (Iceberg/Delta/Hudi) integration.comp-formatsInput/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).Input/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).performancewarmup taskThe task for new ClickHouse team members. Low risk, moderate complexity, no urgency.The task for new ClickHouse team members. Low risk, moderate complexity, no urgency.
Description
Key for cache should be file path + etag (works only for object storages). Similar to metadata cache for Iceberg. Should be implemented at InputFormat implementation for parquet. May cache also column chunks/index pages.
For reference, Iceberg metadata cache: #77156
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
comp-datalakeData lake table formats (Iceberg/Delta/Hudi) integration.Data lake table formats (Iceberg/Delta/Hudi) integration.comp-formatsInput/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).Input/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).performancewarmup taskThe task for new ClickHouse team members. Low risk, moderate complexity, no urgency.The task for new ClickHouse team members. Low risk, moderate complexity, no urgency.