Repository navigation
Commit 9547b09
fix: use session timezone for timestamp subtraction and comparison (apache#25094)
## Which issue does this PR close?
- Closes apache#13212.
## Rationale for this change
Mixed timezone-aware and timezone-naive timestamp operations ignore the
configured session timezone when they interpret the naive value.
For example, use midnight on `2026-07-01` in `America/New_York` and a
naive midnight timestamp on the same date. Set the session timezone to
`+08:00`. They currently compare equal and subtract to zero. The naive
value represents midnight in the session timezone, so the values must
compare unequal and differ by twelve hours.
## What changes are included in this PR?
This PR:
- Casts both operands of a mixed timestamp pair to the session timezone
and a common timestamp precision.
- Uses the same rule for subtraction and comparisons, so `a = b` and `a
- b = 0` agree.
- Applies comparison coercion to binary comparisons, `IN`, `BETWEEN`,
simple `CASE`, `IN` subqueries, and `ANY`/`ALL`.
- Applies comparison coercion recursively through dictionary, run-end
encoded, list, struct, and map types.
- Passes the session timezone through nested subqueries and
`ExprSimplifier::coerce`, including the
`SessionContext::create_physical_expr` path.
The coerced timestamps use the session timezone. Casting the aware
operand preserves its instant. Without a session timezone, both
comparison and subtraction retain current `main` behavior and use the
aware operand's timezone.
The existing DST-boundary limitation in apache#25084 remains. The implicit
cast can encounter that limitation for ambiguous or nonexistent local
times.
## What is the testing strategy for this PR?
Unit tests, SQL logic tests, and a direct physical-expression test
cover:
- Both operand orders, equal and different precisions, and configured
and unset session timezones.
- Comparisons, subtraction, nested subqueries, and encoded or nested
types.
- Different session and operand timezones, including EST and EDT
offsets.
- The configuration description in `information_schema`.
The merge preserves upstream's regression tests for subtraction without
a session timezone at every timestamp precision.
## Are there any user-facing changes?
Yes:
- Mixed timestamp subtraction and comparison results change when
`datafusion.execution.time_zone` is set.
- New public APIs: `BinaryTypeCoercer::with_session_time_zone` and
`comparison_coercion_with_session_timezone`.
- Deprecated helpers: `get_coerce_type_for_list` and
`get_coerce_type_for_case_when`. Both remain available with their
existing behavior.
The configuration documentation and the 56.0.0 upgrade guide describe
these changes.
---------
Co-authored-by: Claude Opus 5 <[email protected]>1 parent 6047791 commit 9547b09
14 files changed
Lines changed: 768 additions & 98 deletions
File tree
- datafusion
- common/src
- core/tests/expr_api
- expr-common/src/type_coercion
- binary/tests
- expr/src/type_coercion
- optimizer/src
- analyzer
- simplify_expressions
- sqllogictest/test_files
- datetime
- docs/source
- library-user-guide/upgrading
- user-guide
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
943 | 943 | | |
944 | 944 | | |
945 | 945 | | |
946 | | - | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
947 | 949 | | |
948 | 950 | | |
949 | 951 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
| |||
364 | 365 | | |
365 | 366 | | |
366 | 367 | | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
367 | 405 | | |
368 | 406 | | |
369 | 407 | | |
| |||
0 commit comments