Repository navigation
Commit 34f922d
feat: add GroupColumn support for Duration in multi-column GROUP BY (apache#23783)
## Which issue does this PR close?
- Part of apache#22715.
## Rationale for this change
`multi_group_by::group_column_supported_type` gates which GROUP BY
columns may
use the column-wise `GroupValuesColumn` fast path, and the gate is
all-or-nothing: a single unsupported column forces the **entire**
grouping onto
the byte-encoded `GroupValuesRows` fallback, even when every other key
column
would have qualified. A `Duration` key triggers exactly that today, so
an
otherwise-qualifying multi-column `GROUP BY` pays the row-encoding tax
because of
one column.
`Duration` shares the `i64` native representation already used by
`Timestamp`,
so supporting it is a pure slot-in of the existing
`PrimitiveGroupValueBuilder` —
no new builder type and no new comparison/hash logic.
## What changes are included in this PR?
- Accept `Duration(_)` in `group_column_supported_type` (all four
`TimeUnit`s are
valid Arrow types, unlike the restricted `Time32`/`Time64` set).
- Dispatch the four `Duration*Type` units in `make_group_column`.
- Extend the `group_column_supported_type` ↔ `make_group_column`
consistency fuzz
with all four Duration units.
- Add a `(Duration, Int32)` group-count benchmark to
`benches/multi_group_by.rs`.
## Are these changes tested?
Yes.
- New unit test `test_group_values_column_duration`: a
`(Duration(Microsecond), Int64)` key stays on the `GroupValuesColumn`
path,
dedups on the composite key (including the `(null, null)` pair), and
round-trips with the `Duration` output type preserved (not the bare
`i64`).
- The consistency fuzz now asserts every `Duration` unit routes through
the
dispatcher.
- New single- and multi-column `Duration` `GROUP BY` coverage in
`group_by.slt`.
## Are there any user-facing changes?
No API changes. `GROUP BY` queries with a `Duration` key now use the
column-wise
fast path instead of the row-encoded fallback; results are unchanged.
---------
Co-authored-by: tohuya6 <[email protected]>1 parent 1305241 commit 34f922d
3 files changed
Lines changed: 195 additions & 9 deletions
File tree
- datafusion
- physical-plan
- benches
- src/aggregates/group_values/multi_group_by
- sqllogictest/test_files
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
31 | 31 | | |
32 | | - | |
| 32 | + | |
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
| |||
444 | 444 | | |
445 | 445 | | |
446 | 446 | | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
447 | 531 | | |
448 | 532 | | |
449 | 533 | | |
| |||
453 | 537 | | |
454 | 538 | | |
455 | 539 | | |
| 540 | + | |
456 | 541 | | |
457 | 542 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
43 | 44 | | |
44 | 45 | | |
45 | 46 | | |
| |||
963 | 964 | | |
964 | 965 | | |
965 | 966 | | |
| 967 | + | |
966 | 968 | | |
967 | 969 | | |
968 | 970 | | |
| |||
1042 | 1044 | | |
1043 | 1045 | | |
1044 | 1046 | | |
| 1047 | + | |
| 1048 | + | |
| 1049 | + | |
| 1050 | + | |
| 1051 | + | |
| 1052 | + | |
| 1053 | + | |
| 1054 | + | |
| 1055 | + | |
| 1056 | + | |
| 1057 | + | |
| 1058 | + | |
| 1059 | + | |
| 1060 | + | |
1045 | 1061 | | |
1046 | 1062 | | |
1047 | 1063 | | |
| |||
1278 | 1294 | | |
1279 | 1295 | | |
1280 | 1296 | | |
1281 | | - | |
| 1297 | + | |
| 1298 | + | |
| 1299 | + | |
| 1300 | + | |
1282 | 1301 | | |
1283 | 1302 | | |
1284 | 1303 | | |
| |||
1577 | 1596 | | |
1578 | 1597 | | |
1579 | 1598 | | |
| 1599 | + | |
| 1600 | + | |
| 1601 | + | |
| 1602 | + | |
1580 | 1603 | | |
1581 | 1604 | | |
1582 | 1605 | | |
| |||
1620 | 1643 | | |
1621 | 1644 | | |
1622 | 1645 | | |
| 1646 | + | |
| 1647 | + | |
| 1648 | + | |
| 1649 | + | |
| 1650 | + | |
| 1651 | + | |
| 1652 | + | |
| 1653 | + | |
| 1654 | + | |
| 1655 | + | |
| 1656 | + | |
| 1657 | + | |
| 1658 | + | |
| 1659 | + | |
| 1660 | + | |
| 1661 | + | |
| 1662 | + | |
| 1663 | + | |
| 1664 | + | |
| 1665 | + | |
| 1666 | + | |
| 1667 | + | |
| 1668 | + | |
| 1669 | + | |
| 1670 | + | |
| 1671 | + | |
| 1672 | + | |
| 1673 | + | |
| 1674 | + | |
| 1675 | + | |
| 1676 | + | |
| 1677 | + | |
| 1678 | + | |
| 1679 | + | |
| 1680 | + | |
| 1681 | + | |
| 1682 | + | |
| 1683 | + | |
| 1684 | + | |
| 1685 | + | |
| 1686 | + | |
| 1687 | + | |
| 1688 | + | |
| 1689 | + | |
| 1690 | + | |
| 1691 | + | |
| 1692 | + | |
| 1693 | + | |
| 1694 | + | |
| 1695 | + | |
| 1696 | + | |
1623 | 1697 | | |
1624 | 1698 | | |
1625 | 1699 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5694 | 5694 | | |
5695 | 5695 | | |
5696 | 5696 | | |
| 5697 | + | |
| 5698 | + | |
| 5699 | + | |
| 5700 | + | |
| 5701 | + | |
| 5702 | + | |
| 5703 | + | |
| 5704 | + | |
| 5705 | + | |
| 5706 | + | |
| 5707 | + | |
| 5708 | + | |
| 5709 | + | |
| 5710 | + | |
| 5711 | + | |
| 5712 | + | |
| 5713 | + | |
| 5714 | + | |
| 5715 | + | |
| 5716 | + | |
| 5717 | + | |
| 5718 | + | |
| 5719 | + | |
| 5720 | + | |
| 5721 | + | |
| 5722 | + | |
| 5723 | + | |
0 commit comments