Repository navigation
Commit c0ec512
perf: don't build a parquet stream for a file range with no row group left (apache#26476)
I've got a few fixes to upstream so just starting with small ones.
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes apache#26475
## Rationale for this change
If row-group statistics or Bloom filters prune every row group in a file
range, the opener still builds the decoder read plans, the decoder and
the row filter, then hands back a stream morsel that yields nothing.
`PruneFile` already ends early when file statistics prune the range;
this does the same once row-group pruning has.
## What changes are included in this PR?
`PruneWithStatistics` and `PruneWithBloomFilters` return
`ParquetOpenState::Done` when no row group survives.
## What is the testing strategy for this PR?
Testing to ensure that we do actually skip them!
## Are there any user-facing changes?
Nope!
---------
Co-authored-by: Andrew Lamb <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>1 parent d090d29 commit c0ec512
2 files changed
Lines changed: 96 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
660 | 660 | | |
661 | 661 | | |
662 | 662 | | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
663 | 666 | | |
664 | 667 | | |
665 | 668 | | |
| |||
690 | 693 | | |
691 | 694 | | |
692 | 695 | | |
693 | | - | |
694 | | - | |
695 | | - | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
696 | 703 | | |
697 | 704 | | |
698 | 705 | | |
| |||
2690 | 2697 | | |
2691 | 2698 | | |
2692 | 2699 | | |
| 2700 | + | |
| 2701 | + | |
| 2702 | + | |
| 2703 | + | |
| 2704 | + | |
| 2705 | + | |
| 2706 | + | |
| 2707 | + | |
| 2708 | + | |
| 2709 | + | |
| 2710 | + | |
| 2711 | + | |
| 2712 | + | |
| 2713 | + | |
| 2714 | + | |
| 2715 | + | |
| 2716 | + | |
| 2717 | + | |
| 2718 | + | |
| 2719 | + | |
2693 | 2720 | | |
2694 | 2721 | | |
2695 | 2722 | | |
| |||
3234 | 3261 | | |
3235 | 3262 | | |
3236 | 3263 | | |
| 3264 | + | |
| 3265 | + | |
| 3266 | + | |
| 3267 | + | |
| 3268 | + | |
| 3269 | + | |
| 3270 | + | |
| 3271 | + | |
| 3272 | + | |
| 3273 | + | |
| 3274 | + | |
| 3275 | + | |
| 3276 | + | |
| 3277 | + | |
| 3278 | + | |
| 3279 | + | |
| 3280 | + | |
| 3281 | + | |
| 3282 | + | |
| 3283 | + | |
| 3284 | + | |
| 3285 | + | |
| 3286 | + | |
| 3287 | + | |
| 3288 | + | |
| 3289 | + | |
| 3290 | + | |
| 3291 | + | |
| 3292 | + | |
| 3293 | + | |
| 3294 | + | |
| 3295 | + | |
| 3296 | + | |
| 3297 | + | |
| 3298 | + | |
| 3299 | + | |
| 3300 | + | |
| 3301 | + | |
| 3302 | + | |
| 3303 | + | |
| 3304 | + | |
3237 | 3305 | | |
3238 | 3306 | | |
3239 | 3307 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
102 | 108 | | |
103 | 109 | | |
104 | 110 | | |
| |||
706 | 712 | | |
707 | 713 | | |
708 | 714 | | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
709 | 734 | | |
710 | 735 | | |
711 | 736 | | |
| |||
0 commit comments