You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 1b27fc1
Browse filesBrowse the repository at this point in the historyBrowse files
fix: never fix the AT TIME ZONE branch before the input type is final
`timezone` lowered itself to `CAST` / `to_local_time(CAST)` in `simplify`.
That fixed the branch from the type the simplifier saw, and that type is
not always final:
- `PREPARE` runs the optimizer without the analyzer, so a `CASE` of a
naive and an aware column is still uncoerced there.
- An untyped placeholder has no type until `EXECUTE`.
Remove the lowering. The branch is now chosen only from the argument
type, in `return_field_from_args` and `invoke_with_args`. The SQL planner
no longer falls back to reading the input type when no `ExprPlanner`
handles `AT TIME ZONE`. It returns an error, as for `EXTRACT`.
Without the `CAST`, `unwrap_cast_in_comparison` no longer relabels a
literal compared with `naive AT TIME ZONE 'zone'` (#25095),
so one existing expectation loses a row that never matched.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
@@ -34,17 +32,20 @@ use datafusion_macros::user_doc;
34
32
///
35
33
/// The operator always returns the *other* kind of timestamp:
36
34
///
37
-
/// | Input type (after coercion) | Result | Plan after simplification |
35
+
/// | Input type | Result | Computed as |
38
36
/// | --- | --- | --- |
39
37
/// | `Timestamp(unit, Some(_))` (aware) | the wall clock of the instant in `zone`, as `Timestamp(unit, None)` | `to_local_time(CAST(expression AS Timestamp(unit, Some(zone))))` |
40
38
/// | `Timestamp(unit, None)` (naive) | the value read as a wall clock in `zone`, as `Timestamp(unit, Some(zone))` | `CAST(expression AS Timestamp(unit, Some(zone)))` |
41
39
/// | any other type | as for a naive input | `CAST(expression AS Timestamp(ns, Some(zone)))` |
42
40
///
43
-
/// The choice needs the input type *after* type coercion. For example, a
44
-
/// `CASE` with a naive arm and an aware arm is aware only after coercion. The
45
-
/// SQL planner runs before coercion, so it plans `AT TIME ZONE` as a call to
46
-
/// this function (see `DatetimeFunctionPlanner`), and [`Self::simplify`]
47
-
/// lowers the call once the type is known.
41
+
/// The choice depends only on the type of `expression`, and it is made every
42
+
/// time that type is read: [`ScalarUDFImpl::return_field_from_args`] for the
43
+
/// plan, and [`ScalarUDFImpl::invoke_with_args`] for the data. The function is
44
+
/// deliberately never rewritten into the `CAST` form during planning. A rewrite
45
+
/// would fix the choice from the type at that moment, and that type is not
46
+
/// always final: the SQL planner runs before type coercion, `PREPARE` runs the
47
+
/// optimizer without the analyzer, and an untyped placeholder has no type
48
+
/// until `EXECUTE`.
48
49
///
49
50
/// `zone` must be a constant, because it determines the result type.
50
51
#[user_doc(
@@ -107,8 +108,8 @@ impl TimezoneFunc {
107
108
}
108
109
}
109
110
110
-
/// The two steps that `timezone(zone, <input_type>)` lowers to: the target
111
-
/// type of the `CAST`, and whether `to_local_time` follows it.
111
+
/// The two steps that `timezone(zone, <input_type>)` does: the target type of
112
+
/// the `CAST`, and whether `to_local_time` follows it.
select to_local_time('2024-04-01T00:00:20Z'::timestamp AT TIME ZONE 'Europe/timezone');
4447
4447
4448
4448
# valid query
@@ -4645,10 +4645,14 @@ SELECT column1 FROM t_utc WHERE column1 < '2024-02-01T00:00:00' AT TIME ZONE 'Am
4645
4645
2024-01-01T00:00:01Z
4646
4646
2024-02-01T00:00:01Z
4647
4647
4648
+
# No row matches: the row `2024-02-01T00:00:01+01:00` is 2024-01-31T23:00:01Z,
4649
+
# and the literal is 2024-02-01T00:00:01Z. DuckDB also returns no row. Before
4650
+
# `AT TIME ZONE` became the `timezone` function, it was a `CAST`, and
4651
+
# `unwrap_cast_in_comparison` relabelled the literal instead of converting it
4652
+
# (https://github.com/apache/datafusion/issues/25095), so this row matched.
4648
4653
query P
4649
4654
SELECT column1 FROM t_europe WHERE column1 = '2024-01-31T16:00:01' AT TIME ZONE 'America/Los_Angeles';
4650
4655
----
4651
-
2024-02-01T00:00:01+01:00
4652
4656
4653
4657
query P
4654
4658
SELECT column1 FROM t_europe WHERE column1 BETWEEN '2020-01-01T00:00:00' AT TIME ZONE 'Australia/Brisbane' AND '2024-02-01T00:00:00' AT TIME ZONE 'America/Los_Angeles';
@@ -5964,30 +5968,61 @@ FROM (SELECT arrow_cast(naive, 'Dictionary(Int32, Timestamp(Nanosecond, None))')
# The plan lowers to CAST / to_local_time after simplification. The column is
5982
-
# named with the operator syntax.
5989
+
statement ok
5990
+
PREPARE at_tz_case_p(boolean) AS
5991
+
SELECT arrow_typeof((CASE WHEN b THEN naive ELSE aware END) AT TIME ZONE 'America/Denver'),
5992
+
(CASE WHEN b THEN naive ELSE aware END) AT TIME ZONE 'America/Denver'
5993
+
FROM at_tz_case WHERE b = $1
5994
+
5995
+
query TP
5996
+
EXECUTE at_tz_case_p(true)
5997
+
----
5998
+
Timestamp(ns) 2024-01-01T05:00:00
5999
+
6000
+
statement ok
6001
+
DEALLOCATE at_tz_case_p
6002
+
6003
+
# A view over such a CASE
6004
+
statement ok
6005
+
CREATE VIEW at_tz_case_v AS SELECT CASE WHEN b THEN naive ELSE aware END AS x FROM at_tz_case
6006
+
6007
+
query TP
6008
+
SELECT arrow_typeof(x AT TIME ZONE 'America/Denver'), x AT TIME ZONE 'America/Denver' FROM at_tz_case_v
6009
+
----
6010
+
Timestamp(ns) 2024-01-01T05:00:00
6011
+
6012
+
statement ok
6013
+
DROP VIEW at_tz_case_v
6014
+
6015
+
# `AT TIME ZONE` stays a `timezone` call in the plan. It is not rewritten into
6016
+
# a `CAST` during planning, because the type at that point is not always final.
6017
+
# The column is named with the operator syntax.
5983
6018
query TT
5984
6019
EXPLAIN SELECT (CASE WHEN b THEN naive ELSE aware END) AT TIME ZONE 'America/Denver', naive AT TIME ZONE 'America/Denver' FROM at_tz_case
5985
6020
----
5986
6021
logical_plan
5987
-
01)Projection: to_local_time(CAST(CASE WHEN at_tz_case.b THEN CAST(at_tz_case.naive AS Timestamp(ns, "UTC")) ELSE at_tz_case.aware END AS Timestamp(ns, "America/Denver"))) AS CASE WHEN at_tz_case.b THEN at_tz_case.naive ELSE at_tz_case.aware END AT TIME ZONE 'America/Denver', CAST(at_tz_case.naive AS Timestamp(ns, "America/Denver")) AS at_tz_case.naive AT TIME ZONE 'America/Denver'
6022
+
01)Projection: timezone(Utf8("America/Denver"), CASE WHEN at_tz_case.b THEN CAST(at_tz_case.naive AS Timestamp(ns, "UTC")) ELSE at_tz_case.aware END), timezone(Utf8("America/Denver"), at_tz_case.naive)
01)ProjectionExec: expr=[to_local_time(CAST(CASE WHEN b@2 THEN CAST(naive@1 AS Timestamp(ns, "UTC")) ELSE aware@0 END AS Timestamp(ns, "America/Denver"))) as CASE WHEN at_tz_case.b THEN at_tz_case.naive ELSE at_tz_case.aware END AT TIME ZONE 'America/Denver', CAST(naive@1 AS Timestamp(ns, "America/Denver")) as at_tz_case.naive AT TIME ZONE 'America/Denver']
6025
+
01)ProjectionExec: expr=[timezone(America/Denver, CASE WHEN b@2 THEN CAST(naive@1 AS Timestamp(ns, "UTC")) ELSE aware@0 END) as CASE WHEN at_tz_case.b THEN at_tz_case.naive ELSE at_tz_case.aware END AT TIME ZONE 'America/Denver', timezone(America/Denver, naive@1) as at_tz_case.naive AT TIME ZONE 'America/Denver']
0 commit comments