Repository navigation
Commit 2784001
committed
feat(parquet): decline pushdown when projection has few non-filter columns
RowFilter has a fixed per-row machinery overhead that only pays for
itself when the wide-column decode it lets us skip is meaningful.
When the projection is narrow and the filter columns already cover
most of it (typical for GROUP BY col-style queries), the overhead
dominates and pushdown regresses.
Add a plan-time heuristic in ParquetSource::try_pushdown_filters that
declines pushdown when the projection contains fewer than 3 columns
not referenced by the filter. When declined, the filter stays above
the scan in a FilterExec (correctness preserved) and the predicate is
still injected into ParquetSource for stats / bloom / page-index
pruning.
Kept intentionally simple — a plan-time column-count check with no
tuning knob. Complexity budget is reserved for the runtime
adaptive-placement work tracked in apache#22883.
Part of apache#3463.1 parent fa40708 commit 2784001
1 file changed
Lines changed: 38 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
| 49 | + | |
49 | 50 | | |
50 | 51 | | |
51 | 52 | | |
| |||
827 | 828 | | |
828 | 829 | | |
829 | 830 | | |
830 | | - | |
| 831 | + | |
| 832 | + | |
| 833 | + | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
831 | 868 | | |
832 | 869 | | |
833 | 870 | | |
| |||
0 commit comments