Repository navigation
Commit 07f24b6
committed
Treat an unanswered groups accumulator question as no permission
`AggregateUDFImpl::groups_accumulator_supported_for_types` returned `bool`
with a `false` default, and `rewrite_pays_for_count` read `false` as
"the rewrite pays". Every aggregate that did not override the method was
therefore waved through the gate, which is the permissive answer rather
than the safe one.
Return `Option<bool>` instead, defaulting to `None`, and rewrite only on
`Some(false)`. `None` says the aggregate does not answer the question, and
silence is not evidence that the rewrite pays.
Measured on unmodified upstream, over 4,000,000 rows in 2,000 groups,
`SELECT g, count(*), sum(DISTINCT int_col) FROM t GROUP BY g` reached the
gated path and regressed 3.15x: 71.0 MiB unrewritten against 223.4 MiB
rewritten. `min(DISTINCT x)` reached it too, and regresses much further,
because `min(DISTINCT x)` is `min(x)` and the unrewritten plan keeps one
scalar per group.
`Count` is the only implementor and its answers are unchanged, so no plan
that the gate already allowed changes shape.1 parent b28c289 commit 07f24b6
4 files changed
Lines changed: 199 additions & 25 deletions
File tree
- datafusion
- expr/src
- functions-aggregate/src
- optimizer/src
- sqllogictest/test_files
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
258 | 258 | | |
259 | 259 | | |
260 | 260 | | |
261 | | - | |
| 261 | + | |
262 | 262 | | |
263 | 263 | | |
264 | 264 | | |
| |||
638 | 638 | | |
639 | 639 | | |
640 | 640 | | |
641 | | - | |
642 | | - | |
643 | | - | |
644 | | - | |
645 | | - | |
646 | | - | |
647 | | - | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
648 | 654 | | |
649 | 655 | | |
650 | 656 | | |
651 | 657 | | |
652 | | - | |
653 | | - | |
| 658 | + | |
| 659 | + | |
654 | 660 | | |
655 | 661 | | |
656 | 662 | | |
| |||
1592 | 1598 | | |
1593 | 1599 | | |
1594 | 1600 | | |
1595 | | - | |
| 1601 | + | |
1596 | 1602 | | |
1597 | 1603 | | |
1598 | 1604 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
354 | 354 | | |
355 | 355 | | |
356 | 356 | | |
| 357 | + | |
357 | 358 | | |
358 | 359 | | |
359 | 360 | | |
360 | 361 | | |
361 | 362 | | |
362 | 363 | | |
363 | | - | |
| 364 | + | |
364 | 365 | | |
365 | | - | |
| 366 | + | |
366 | 367 | | |
367 | 368 | | |
368 | | - | |
| 369 | + | |
369 | 370 | | |
370 | 371 | | |
371 | | - | |
| 372 | + | |
372 | 373 | | |
373 | 374 | | |
374 | 375 | | |
| |||
378 | 379 | | |
379 | 380 | | |
380 | 381 | | |
381 | | - | |
| 382 | + | |
382 | 383 | | |
383 | 384 | | |
384 | 385 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
81 | 81 | | |
82 | 82 | | |
83 | 83 | | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
88 | 90 | | |
89 | 91 | | |
90 | 92 | | |
| |||
190 | 192 | | |
191 | 193 | | |
192 | 194 | | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
193 | 208 | | |
194 | 209 | | |
195 | 210 | | |
| |||
202 | 217 | | |
203 | 218 | | |
204 | 219 | | |
205 | | - | |
| 220 | + | |
206 | 221 | | |
207 | 222 | | |
208 | 223 | | |
| |||
446 | 461 | | |
447 | 462 | | |
448 | 463 | | |
| 464 | + | |
449 | 465 | | |
| 466 | + | |
450 | 467 | | |
451 | 468 | | |
452 | 469 | | |
| |||
467 | 484 | | |
468 | 485 | | |
469 | 486 | | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
470 | 546 | | |
471 | 547 | | |
472 | 548 | | |
| |||
808 | 884 | | |
809 | 885 | | |
810 | 886 | | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
811 | 935 | | |
812 | 936 | | |
813 | 937 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
28 | 30 | | |
29 | 31 | | |
30 | 32 | | |
| |||
109 | 111 | | |
110 | 112 | | |
111 | 113 | | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
112 | 137 | | |
113 | 138 | | |
114 | 139 | | |
| |||
365 | 390 | | |
366 | 391 | | |
367 | 392 | | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
368 | 402 | | |
369 | 403 | | |
370 | 404 | | |
| |||
400 | 434 | | |
401 | 435 | | |
402 | 436 | | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
403 | 446 | | |
404 | 447 | | |
405 | 448 | | |
| |||
0 commit comments