Repository navigation
Commit efd36e7
authored
fix: preserve date_trunc sorting for timezone-aware timestamps (apache#25409)
## Which issue does this PR close?
- Closes apache#25353.
## Rationale for this change
`ORDER BY date_trunc(...)` can return incorrectly ordered results across
named timezone transitions because the optimizer incorrectly assumes
that truncation preserves ordering and removes a required sort.
## What changes are included in this PR?
Only propagate input ordering when the precision is constant and the
input range type is a timezone-free timestamp, `Time32`, or `Time64`.
Return `SortProperties::Unordered` for timezone-aware timestamps and
unknown range types. An unknown range can hide a timezone-aware result,
such as `from_unixtime(c, 'America/Goose_Bay')`.
Follow-up optimizations are tracked separately:
- apache#25464: recognize safe timezone-aware cases, such as UTC and fixed
offsets.
- apache#25465: provide typed bounds for `date_bin` and `from_unixtime`.
## What is the testing strategy for this PR?
- A unit test covers timezone-free, timezone-aware, and unknown range
types.
- SQL regression tests cover the original `America/Goose_Bay` reproducer
and the `from_unixtime` case identified during review.
- The new unknown-range assertion and `from_unixtime` regression fail
with the previous ordering check and pass with the allowlist.
- Formatting, full Clippy, and extended workspace tests pass locally.
## Are there any user-facing changes?
Affected queries return correctly ordered results.
Some queries may retain additional sorts, including UTC and fixed-offset
timestamps, and timezone-free expressions whose bounds analysis does not
provide their output type.1 parent 696dd18 commit efd36e7
2 files changed
Lines changed: 96 additions & 4 deletions
File tree
- datafusion
- functions/src/datetime
- sqllogictest/test_files/datetime
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
431 | 431 | | |
432 | 432 | | |
433 | 433 | | |
434 | | - | |
435 | 434 | | |
436 | 435 | | |
437 | 436 | | |
438 | | - | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
439 | 443 | | |
440 | 444 | | |
441 | 445 | | |
| |||
889 | 893 | | |
890 | 894 | | |
891 | 895 | | |
892 | | - | |
893 | 896 | | |
| 897 | + | |
894 | 898 | | |
895 | 899 | | |
896 | 900 | | |
| 901 | + | |
| 902 | + | |
897 | 903 | | |
898 | 904 | | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
| 937 | + | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
899 | 945 | | |
900 | 946 | | |
901 | 947 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1655 | 1655 | | |
1656 | 1656 | | |
1657 | 1657 | | |
1658 | | - | |
| 1658 | + | |
| 1659 | + | |
| 1660 | + | |
| 1661 | + | |
| 1662 | + | |
| 1663 | + | |
| 1664 | + | |
| 1665 | + | |
| 1666 | + | |
| 1667 | + | |
| 1668 | + | |
| 1669 | + | |
| 1670 | + | |
| 1671 | + | |
| 1672 | + | |
| 1673 | + | |
| 1674 | + | |
| 1675 | + | |
| 1676 | + | |
| 1677 | + | |
| 1678 | + | |
| 1679 | + | |
| 1680 | + | |
| 1681 | + | |
| 1682 | + | |
| 1683 | + | |
| 1684 | + | |
| 1685 | + | |
| 1686 | + | |
| 1687 | + | |
| 1688 | + | |
| 1689 | + | |
| 1690 | + | |
| 1691 | + | |
| 1692 | + | |
| 1693 | + | |
| 1694 | + | |
| 1695 | + | |
| 1696 | + | |
| 1697 | + | |
| 1698 | + | |
| 1699 | + | |
| 1700 | + | |
| 1701 | + | |
| 1702 | + | |
| 1703 | + | |
| 1704 | + | |
1659 | 1705 | | |
1660 | 1706 | | |
1661 | 1707 | | |
| |||
0 commit comments