Improve compatibility with non-whole-minute timezone offsets#27080
Improve compatibility with non-whole-minute timezone offsets#27080alexey-milovidov merged 8 commits intoClickHouse:masterfrom
Conversation
|
2 things in CI:
Let's see if a new commit fixing (2) also shows the address issues or it has been something temporary. |
That's interesting... |
4387ea7 to
1b58e27
Compare
|
Rebased with master. No new changes (it was clean already) |
|
We have 2x performance degradation of |
base/common/DateLUTImpl.h
Outdated
There was a problem hiding this comment.
How the opposite is possible if we are dividing the number of seconds after midnight (always positive)?
There was a problem hiding this comment.
I was probably thinking about negative values of t (with Datetime64), which will lead to negative time and negative res; but that doesn't see to be supported anyway.
It might be because |
|
I'm not sure about this change. ... but if we can avoid any performance degradation. |
|
I think the performance degradation can be solved in a similar way as the whole hour thing, by adding another flag like |
|
Ok, sounds good. |
1b58e27 to
0771329
Compare
|
Rebased and applied the improvements from the comments. Performance of the affected query:
Looking at how things work internally (DateTimeTransforms.h), it looks like the fast path (common offsets) of some of the functions (toSecond, toMinute) could be vectorized using the same architecture as other functions (accepting an array instead of a single element), but that's something for a different time. |
|
Ok. Theoretically it can also be auto-vectorized, because the function is inlined. But it depends if compiler will be able to move "unlikely" branch check out of the loop. |
|
I added an invalid optimization (ok for minutes, not for hours). I'm looking into it and will try to unify calls/code if possible. Sadly, changing DateLutImpl.h requires recompiling ~1300 files so it takes time to verify and benchmark |
Instead of mixing multiple rounding, use toStartOfMinuteInterval for all minute related calculations
Looks like GitHub API is down. |
|
@Mergifyio update |
|
Command
|
|
@Mergifyio update |
|
Command
|
alexey-milovidov
left a comment
There was a problem hiding this comment.
Wow, everything is alright :)
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Improve compatibility with non-whole-minute timezone offsets
Detailed description / Documentation draft:
DateLUT was assuming timezone offsets are whole minutes but that's not true, for example Liberia used to be
-0:43:08and later became-0:44:30(it currently is 00:00, but that's less exciting).Closes #27079
It appeared as part of the CI (in #26978)