-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Closed
Labels
enhancementAny new improvement worthy of a entry in the changelogAny new improvement worthy of a entry in the changelog
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
follow #1705
Goal:
Index granularity - per page (used row Group).
Make selective scans on both sorting & non-sorting columns efficient(maybe).
No additional I/O & deserialization costs for non-selective filter.
Describe the solution you'd like
There is already store the PageIndex Index just above the footer. Store offset and length in col metadata.
Using these index with Row Alignment filter page without decode+decompress.
- Add read options for column index based filtering
- Support reading
PageIndexfrom column metadata #1761 - Add function for row alignment with page mask #1790
- Enable column_page_reader read specific row ranges record #1792
- work with arrow-datafusion
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementAny new improvement worthy of a entry in the changelogAny new improvement worthy of a entry in the changelog