Repository navigation
Commit 22095cd
committed
feat: reorder row groups by statistics during sort pushdown
When sort pushdown is active, reorder row groups within each file by
their min/max statistics to match the requested sort order. This helps
TopK queries find optimal values first via dynamic filter pushdown.
- Add reorder_by_statistics to PreparedAccessPlan that sorts
row_group_indexes by the first sort column's min values
- Pass sort order from ParquetSource::try_pushdown_sort through to
the opener via sort_order_for_reorder field
- Reorder happens after pruning but before reverse (they compose)
- Gracefully skips reorder when statistics unavailable, sort expr
is not a simple column, row_selection present, or <=1 row groups
Closes apache#213171 parent d68373e commit 22095cd
3 files changed
Lines changed: 134 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
19 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
20 | 25 | | |
21 | 26 | | |
22 | 27 | | |
| |||
377 | 382 | | |
378 | 383 | | |
379 | 384 | | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
380 | 485 | | |
381 | 486 | | |
382 | 487 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
| 57 | + | |
57 | 58 | | |
58 | 59 | | |
59 | 60 | | |
| |||
154 | 155 | | |
155 | 156 | | |
156 | 157 | | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
157 | 162 | | |
158 | 163 | | |
159 | 164 | | |
| |||
304 | 309 | | |
305 | 310 | | |
306 | 311 | | |
| 312 | + | |
307 | 313 | | |
308 | 314 | | |
309 | 315 | | |
| |||
752 | 758 | | |
753 | 759 | | |
754 | 760 | | |
| 761 | + | |
755 | 762 | | |
756 | 763 | | |
757 | 764 | | |
| |||
1223 | 1230 | | |
1224 | 1231 | | |
1225 | 1232 | | |
| 1233 | + | |
| 1234 | + | |
| 1235 | + | |
| 1236 | + | |
| 1237 | + | |
| 1238 | + | |
| 1239 | + | |
| 1240 | + | |
| 1241 | + | |
| 1242 | + | |
1226 | 1243 | | |
1227 | 1244 | | |
1228 | 1245 | | |
| |||
1731 | 1748 | | |
1732 | 1749 | | |
1733 | 1750 | | |
| 1751 | + | |
1734 | 1752 | | |
1735 | 1753 | | |
1736 | 1754 | | |
| |||
1759 | 1777 | | |
1760 | 1778 | | |
1761 | 1779 | | |
| 1780 | + | |
1762 | 1781 | | |
1763 | 1782 | | |
1764 | 1783 | | |
| |||
1785 | 1804 | | |
1786 | 1805 | | |
1787 | 1806 | | |
| 1807 | + | |
1788 | 1808 | | |
1789 | 1809 | | |
1790 | 1810 | | |
| |||
1900 | 1920 | | |
1901 | 1921 | | |
1902 | 1922 | | |
| 1923 | + | |
1903 | 1924 | | |
1904 | 1925 | | |
1905 | 1926 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | | - | |
| 58 | + | |
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| |||
294 | 294 | | |
295 | 295 | | |
296 | 296 | | |
| 297 | + | |
| 298 | + | |
297 | 299 | | |
298 | 300 | | |
299 | 301 | | |
| |||
319 | 321 | | |
320 | 322 | | |
321 | 323 | | |
| 324 | + | |
322 | 325 | | |
323 | 326 | | |
324 | 327 | | |
| |||
570 | 573 | | |
571 | 574 | | |
572 | 575 | | |
| 576 | + | |
573 | 577 | | |
574 | 578 | | |
575 | 579 | | |
| |||
817 | 821 | | |
818 | 822 | | |
819 | 823 | | |
820 | | - | |
| 824 | + | |
| 825 | + | |
| 826 | + | |
821 | 827 | | |
822 | 828 | | |
823 | 829 | | |
| |||
0 commit comments