You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8c762b9
Browse filesBrowse the repository at this point in the historyBrowse files
fix: never fix the AT TIME ZONE branch before the input type is final
`timezone` lowered itself to `CAST` / `to_local_time(CAST)` in `simplify`.
That fixed the branch from the type the simplifier saw, and that type is
not always final:
- `PREPARE` runs the optimizer without the analyzer, so a `CASE` of a
naive and an aware column is still uncoerced there.
- An untyped placeholder has no type until `EXECUTE`.
Remove the lowering. The branch is now chosen only from the argument
type, in `return_field_from_args` and `invoke_with_args`. The SQL planner
no longer falls back to reading the input type when no `ExprPlanner`
handles `AT TIME ZONE`. It returns an error, as for `EXTRACT`.
Without the `CAST`, `unwrap_cast_in_comparison` no longer relabels a
literal compared with `naive AT TIME ZONE 'zone'` (apache#25095),
so one existing expectation loses a row that never matched.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
@@ -34,17 +32,20 @@ use datafusion_macros::user_doc;
34
32
///
35
33
/// The operator always returns the *other* kind of timestamp:
36
34
///
37
-
/// | Input type (after coercion) | Result | Plan after simplification |
35
+
/// | Input type | Result | Computed as |
38
36
/// | --- | --- | --- |
39
37
/// | `Timestamp(unit, Some(_))` (aware) | the wall clock of the instant in `zone`, as `Timestamp(unit, None)` | `to_local_time(CAST(expression AS Timestamp(unit, Some(zone))))` |
40
38
/// | `Timestamp(unit, None)` (naive) | the value read as a wall clock in `zone`, as `Timestamp(unit, Some(zone))` | `CAST(expression AS Timestamp(unit, Some(zone)))` |
41
39
/// | any other type | as for a naive input | `CAST(expression AS Timestamp(ns, Some(zone)))` |
42
40
///
43
-
/// The choice needs the input type *after* type coercion. For example, a
44
-
/// `CASE` with a naive arm and an aware arm is aware only after coercion. The
45
-
/// SQL planner runs before coercion, so it plans `AT TIME ZONE` as a call to
46
-
/// this function (see `DatetimeFunctionPlanner`), and [`Self::simplify`]
47
-
/// lowers the call once the type is known.
41
+
/// The choice depends only on the type of `expression`, and it is made every
42
+
/// time that type is read: [`ScalarUDFImpl::return_field_from_args`] for the
43
+
/// plan, and [`ScalarUDFImpl::invoke_with_args`] for the data. The function is
44
+
/// deliberately never rewritten into the `CAST` form during planning. A rewrite
45
+
/// would fix the choice from the type at that moment, and that type is not
46
+
/// always final: the SQL planner runs before type coercion, `PREPARE` runs the
47
+
/// optimizer without the analyzer, and an untyped placeholder has no type
48
+
/// until `EXECUTE`.
48
49
///
49
50
/// `zone` must be a constant, because it determines the result type.
50
51
#[user_doc(
@@ -107,8 +108,8 @@ impl TimezoneFunc {
107
108
}
108
109
}
109
110
110
-
/// The two steps that `timezone(zone, <input_type>)` lowers to: the target
111
-
/// type of the `CAST`, and whether `to_local_time` follows it.
111
+
/// The two steps that `timezone(zone, <input_type>)` does: the target type of
112
+
/// the `CAST`, and whether `to_local_time` follows it.
select to_local_time('2024-04-01T00:00:20Z'::timestamp AT TIME ZONE 'Europe/timezone');
4336
4336
4337
4337
# valid query
@@ -4534,10 +4534,14 @@ SELECT column1 FROM t_utc WHERE column1 < '2024-02-01T00:00:00' AT TIME ZONE 'Am
4534
4534
2024-01-01T00:00:01Z
4535
4535
2024-02-01T00:00:01Z
4536
4536
4537
+
# No row matches: the row `2024-02-01T00:00:01+01:00` is 2024-01-31T23:00:01Z,
4538
+
# and the literal is 2024-02-01T00:00:01Z. DuckDB also returns no row. Before
4539
+
# `AT TIME ZONE` became the `timezone` function, it was a `CAST`, and
4540
+
# `unwrap_cast_in_comparison` relabelled the literal instead of converting it
4541
+
# (https://github.com/apache/datafusion/issues/25095), so this row matched.
4537
4542
query P
4538
4543
SELECT column1 FROM t_europe WHERE column1 = '2024-01-31T16:00:01' AT TIME ZONE 'America/Los_Angeles';
4539
4544
----
4540
-
2024-02-01T00:00:01+01:00
4541
4545
4542
4546
query P
4543
4547
SELECT column1 FROM t_europe WHERE column1 BETWEEN '2020-01-01T00:00:00' AT TIME ZONE 'Australia/Brisbane' AND '2024-02-01T00:00:00' AT TIME ZONE 'America/Los_Angeles';
@@ -5853,30 +5857,61 @@ FROM (SELECT arrow_cast(naive, 'Dictionary(Int32, Timestamp(Nanosecond, None))')
# The plan lowers to CAST / to_local_time after simplification. The column is
5871
-
# named with the operator syntax.
5878
+
statement ok
5879
+
PREPARE at_tz_case_p(boolean) AS
5880
+
SELECT arrow_typeof((CASE WHEN b THEN naive ELSE aware END) AT TIME ZONE 'America/Denver'),
5881
+
(CASE WHEN b THEN naive ELSE aware END) AT TIME ZONE 'America/Denver'
5882
+
FROM at_tz_case WHERE b = $1
5883
+
5884
+
query TP
5885
+
EXECUTE at_tz_case_p(true)
5886
+
----
5887
+
Timestamp(ns) 2024-01-01T05:00:00
5888
+
5889
+
statement ok
5890
+
DEALLOCATE at_tz_case_p
5891
+
5892
+
# A view over such a CASE
5893
+
statement ok
5894
+
CREATE VIEW at_tz_case_v AS SELECT CASE WHEN b THEN naive ELSE aware END AS x FROM at_tz_case
5895
+
5896
+
query TP
5897
+
SELECT arrow_typeof(x AT TIME ZONE 'America/Denver'), x AT TIME ZONE 'America/Denver' FROM at_tz_case_v
5898
+
----
5899
+
Timestamp(ns) 2024-01-01T05:00:00
5900
+
5901
+
statement ok
5902
+
DROP VIEW at_tz_case_v
5903
+
5904
+
# `AT TIME ZONE` stays a `timezone` call in the plan. It is not rewritten into
5905
+
# a `CAST` during planning, because the type at that point is not always final.
5906
+
# The column is named with the operator syntax.
5872
5907
query TT
5873
5908
EXPLAIN SELECT (CASE WHEN b THEN naive ELSE aware END) AT TIME ZONE 'America/Denver', naive AT TIME ZONE 'America/Denver' FROM at_tz_case
5874
5909
----
5875
5910
logical_plan
5876
-
01)Projection: to_local_time(CAST(CASE WHEN at_tz_case.b THEN CAST(at_tz_case.naive AS Timestamp(ns, "UTC")) ELSE at_tz_case.aware END AS Timestamp(ns, "America/Denver"))) AS CASE WHEN at_tz_case.b THEN at_tz_case.naive ELSE at_tz_case.aware END AT TIME ZONE 'America/Denver', CAST(at_tz_case.naive AS Timestamp(ns, "America/Denver")) AS at_tz_case.naive AT TIME ZONE 'America/Denver'
5911
+
01)Projection: timezone(Utf8("America/Denver"), CASE WHEN at_tz_case.b THEN CAST(at_tz_case.naive AS Timestamp(ns, "UTC")) ELSE at_tz_case.aware END), timezone(Utf8("America/Denver"), at_tz_case.naive)
01)ProjectionExec: expr=[to_local_time(CAST(CASE WHEN b@2 THEN CAST(naive@1 AS Timestamp(ns, "UTC")) ELSE aware@0 END AS Timestamp(ns, "America/Denver"))) as CASE WHEN at_tz_case.b THEN at_tz_case.naive ELSE at_tz_case.aware END AT TIME ZONE 'America/Denver', CAST(naive@1 AS Timestamp(ns, "America/Denver")) as at_tz_case.naive AT TIME ZONE 'America/Denver']
5914
+
01)ProjectionExec: expr=[timezone(America/Denver, CASE WHEN b@2 THEN CAST(naive@1 AS Timestamp(ns, "UTC")) ELSE aware@0 END) as CASE WHEN at_tz_case.b THEN at_tz_case.naive ELSE at_tz_case.aware END AT TIME ZONE 'America/Denver', timezone(America/Denver, naive@1) as at_tz_case.naive AT TIME ZONE 'America/Denver']
0 commit comments