-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Currently we provide a mix of kernels for extracting portions of temporal types, https://docs.rs/arrow-arith/latest/arrow_arith/temporal/index.html
There are a couple of limitations with this implementation:
- The use of generics makes it non-trivial to support types such as Time32/Time64 (Support for extracting hours/minutes/seconds/etc. from
Time32/Time64type in temporal kernels #5261) or intervals (#TBD) - The way the kernels are implemented results in redundant code generation for impossible combinations, e.g. a Date32Array with a DataType of Timestamp
- The use of as_datetime and similar to always proxy via chrono is not only very inefficient, but also hard to follow
- They make use of builders instead of the faster (and more concise)
PrimitiveArray::try_unarymethods - The code in general is quite hard to follow
Describe the solution you'd like
I would like to take a similar approach to used in #4465, where we have a single method that accepts an operation and array, and then dispatches the operation to the specialized implementation for that particular array type. Aside from simplifying the implementation, we could also expose this publicly to facilitate implementation of the SQL extract/date_part function - https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-EXTRACT.
In particular I would like something akin to
pub enum DatePart {
Century,
Year,
...
Minute,
...
DayOfYear,
DayOfWeek,
}
pub fn date_part(array: &dyn Array, part: DatePart) -> Result<Int32Array> {
match array.data_type() {
...
}
}
The existing kernels would be deprecated and updated to act as simple shims to this dynamic dispatch logic, as has been done with the previous datum migrations.
Describe alternatives you've considered
Additional context