string_to_datetime (and Utf8 → Timestamp(tz) casts) fail on DST-ambiguous and nonexistent local times #11039

Status (updated 2026-09-11): this is a blocker for apache/datafusion#25084, not a follow-up to #11037.

#11038 fixes only the timestamp cast kernel. A bare string literal goes through string_to_datetime instead, so these DataFusion queries still fail with #11038 alone, although they are the same query to a user:

SELECT '2024-03-10 02:30:00'::timestamp AT TIME ZONE 'America/New_York';  -- fixed by #11038
SELECT '2024-03-10 02:30:00' AT TIME ZONE 'America/New_York';            -- needs this issue
SET datafusion.execution.time_zone = 'America/New_York';
SELECT '2024-03-10 02:30:00'::timestamptz;                              -- needs this issue
SET datafusion.execution.time_zone = 'America/Sao_Paulo';
SELECT '2018-11-04'::timestamptz;   -- date-only site: local midnight does not exist that day

The fix is #11054, stacked on #11038. It shares #11038's resolution policy (ambiguous resolves to the later instant, a gap shifts forward), so the two paths cannot drift apart.

Describe the bug

string_to_datetime in arrow-cast/src/parse.rs, and therefore every Utf8/LargeUtf8/Utf8View → Timestamp(_, Some(tz)) cast, fails for wall-clock readings that fall on a daylight saving transition of a named timezone:

• the hour repeated by a "fall back" transition (ambiguous), and
• the hour skipped by a "spring forward" transition (nonexistent).

The failure is Error parsing timestamp from '...': error computing timezone offset. Under CastOptions { safe: true } the affected values silently become NULL. It affects both a string without a zone that is interpreted in the target timezone, and a string that carries the zone explicitly ('2024-03-10T02:30:00 America/New_York').

This is the string-parsing sibling of #11037. The cause is the same: three call sites in string_to_datetime resolve the local time with LocalResult::single(), which is None for both LocalResult::Ambiguous and LocalResult::None:

return timezone
    .from_local_datetime(&datetime)
    .single()
    .ok_or_else(|| err("error computing timezone offset"));
// ... (twice more, including for the parsed `Tz` suffix)

To Reproduce

arrow-cast 59.3.0, arrow-array with the chrono-tz feature:

use std::sync::Arc;
use arrow_array::{Array, ArrayRef, StringArray};
use arrow_array::timezone::Tz;
use arrow_cast::parse::string_to_datetime;
use arrow_cast::{cast_with_options, CastOptions};
use arrow_schema::{DataType, TimeUnit};

fn main() {
    let tz: Tz = "America/New_York".parse().unwrap();
    for s in [
        "2024-11-01T00:00:00",                  // unambiguous
        "2024-11-03T01:30:00",                  // ambiguous   (fall back)
        "2024-03-10T02:30:00",                  // nonexistent (spring forward)
        "2024-03-10T02:30:00 America/New_York", // nonexistent, zone given in the string
    ] {
        println!("{s:<40} -> {:?}", string_to_datetime(&tz, s).map(|d| d.to_rfc3339()));
    }

    let strings: ArrayRef = Arc::new(StringArray::from(vec![
        "2024-11-01T00:00:00", "2024-11-03T01:30:00", "2024-03-10T02:30:00",
    ]));
    let to = DataType::Timestamp(TimeUnit::Second, Some("America/New_York".into()));
    let strict = CastOptions { safe: false, ..Default::default() };
    println!("Utf8 -> Timestamp(tz) safe=false: {:?}", cast_with_options(&strings, &to, &strict).map(|a| a.len()));
    let out = cast_with_options(&strings, &to, &CastOptions { safe: true, ..Default::default() }).unwrap();
    let shown: Vec<String> = (0..out.len())
        .map(|i| arrow_cast::display::array_value_to_string(&out, i).unwrap())
        .collect();
    println!("Utf8 -> Timestamp(tz) safe=true:  {shown:?}");
}

2024-11-01T00:00:00                      -> Ok("2024-11-01T00:00:00-04:00")
2024-11-03T01:30:00                      -> Err(ParseError("Error parsing timestamp from '2024-11-03T01:30:00': error computing timezone offset"))
2024-03-10T02:30:00                      -> Err(ParseError("Error parsing timestamp from '2024-03-10T02:30:00': error computing timezone offset"))
2024-03-10T02:30:00 America/New_York     -> Err(ParseError("Error parsing timestamp from '2024-03-10T02:30:00 America/New_York': error computing timezone offset"))
Utf8 -> Timestamp(tz) safe=false: Err(ParseError("Error parsing timestamp from '2024-11-03T01:30:00': error computing timezone offset"))
Utf8 -> Timestamp(tz) safe=true:  ["2024-11-01T00:00:00-04:00", "", ""]

Expected behavior

The same resolution as #11037, which records the PostgreSQL and DuckDB outputs and links to the source of both:

• ambiguous → the later instant (post-transition offset): 2024-11-03T01:30:00 → 2024-11-03T01:30:00-05:00;
• nonexistent → shift forward by the gap (pre-transition offset): 2024-03-10T02:30:00 → 2024-03-10T03:30:00-04:00.

Additional context

• Once Resolve DST-ambiguous and nonexistent local times when casting timestamps to a named timezone #11038 lands, arrow-cast is inconsistent with itself: casting the naive timestamp 2024-03-10T02:30:00 to Timestamp(_, Some("America/New_York")) returns 03:30:00-04:00, while casting the string '2024-03-10T02:30:00' to the same type still errors. Same wall-clock value, same target type, different result depending on the source type.
• In DataFusion this is the difference between '2024-03-10T02:30:00'::timestamp::timestamptz (works after 11038) and '2024-03-10T02:30:00'::timestamptz (still fails) under a named session timezone. DataFusion's timestamps.slt currently asserts that TIMESTAMPTZ '2023-03-12 02:00:00 America/Los_Angeles' is an error, so that expectation flips when this is fixed.
• Kept separate from Casting Timestamp(_, None) to a named timezone fails on DST-ambiguous and nonexistent local times #11037 because string_to_datetime is public API and generic over T: TimeZone, so the fix has a different shape: from_local_datetime(..).latest() for the overlap, and for the gap a probe of from_local_datetime(dt - 24h).earliest() to recover the pre-transition offset followed by from_utc_datetime. Three call sites plus tests for the zone-suffix form.