Repository navigation
Description
Problem statement
Computations dealing with arrays of date/time values in a time zone go by obtaining the Tz value and then calling its methods per element.
This is inefficient for two reasons:
- A per-element dispatch on the
TzInnerenum, which may or may not be optimized out of the loop; - For named time zones, a lookup into the chrono-tz generated dispatch and method call(s), which are again subject to optimization. Some of the time zones, most importantly UTC, amount to applying a fixed offset invariantly to all values.
Candidates for optimization
Extracted and summarized with Claude:
1. date_part extraction (year/month/day/hour/etc.)
arrow-arith/src/temporal.rs:418-488
Four near-identical impls for TimestampSecondType / MillisecondType / MicrosecondType / NanosecondType. tz is resolved once via get_tz(...), then each element does:
timestamp_s_to_datetime(d)
.map(|c| Utc.from_utc_datetime(&c).with_timezone(&tz))
.map(map_func)with_timezone invokes Tz::offset_from_utc_datetime per row.
2. Timestamp ± interval arithmetic
arrow-arith/src/numeric.rs:437 resolves l_tz once, then per-element ops (e.g. Op::Add at numeric.rs:456-465) call T::add_year_month(l, r, l_tz), which forwards to add_year_months in arrow-array/src/types.rs:412-419 → as_datetime_with_timezone (candidate 6) → .with_timezone(&tz).
3. Timestamp display / pretty-print
arrow-cast/src/display.rs:755-763, write_timestamp, invoked once per row by the timestamp_display! macro:
let date = Utc.from_utc_datetime(&naive).with_timezone(&tz);4. Timestamp → Time32/Time64 cast
arrow-cast/src/cast/mod.rs:615-631 (as_time_res_with_timezone), called per element from many (Timestamp(_, tz), Time32/Time64(_)) match arms, e.g. cast/mod.rs:1974-1985. tz is parsed once per cast call, then as_datetime_with_timezone (candidate 6) is called inside try_unary for every element.
5. Timestamp → Date32 cast
arrow-cast/src/cast/mod.rs:633-657 (timestamp_to_date32): tz parsed once, then array.try_unary(|x| as_datetime_with_timezone::<T>(x, tz)...) per element.
6. Timestamp → Timestamp with different timezone
arrow-cast/src/cast/mod.rs:2629-2648 (adjust_timestamp_to_timezone):
let adjust = |o| {
let local = as_datetime::<T>(o)?;
let offset = to_tz.offset_from_local_datetime(&local).single()?;
T::from_naive_datetime(local - offset.fix(), None)
};to_tz is loop-invariant across the whole unary_opt/try_unary call. This is the cleanest optimization target: if to_tz is a fixed offset, offset is also loop-invariant and can be computed once outside the closure.
Proposed solution
Add a public accessor on Tz, e.g. pub fn fixed_offset(&self) -> Option<FixedOffset>, in arrow-array/src/timezone.rs:
- Returns
Someimmediately forTzInner::Offset. - Can also return
Someon recognized named time zone variants, primarilyUTCand its aliases,Etc/GMT*etc..
Each candidate site would then check this once outside its per-element loop and, when Some(offset), apply the offset via plain arithmetic (NaiveDateTime + Duration / integer add) instead of dispatching through chrono::TimeZone per element.
Other aliternatives considered
Replace chrono with jiff as proposed in #9183. Jiff has the requisite method on its TimeZone type.
That is a much wider change and requires API breaks.
Additional context
In testing of a similar optimization done outsize of arrow-rs, elimination of the chrono-tz dispatch from offsetting a timestamp for a timestamp-to-string array cast resulted in 13-18% improvement.
Activity
tustvold commentedon Aug 26, 2026
This is necessary to handle daylight savings time correctly, the offset depends on the values
mzabaluev commentedon Aug 26, 2026
Yes, in the case when the tz does not amount to a fixed offset, the computation still has to go through a
TimeZonemethod call per value.