I am moving a discussion from pangeo over to here: pangeo-data/pangeo#764. xarray has a method on their CFTimeIndex to shift the cftime.
E.g.,
import xarray as xr
times = xr.cftime_range('1950', '1952', freq='AS')
print(times)
>>> CFTimeIndex([1950-01-01 00:00:00, 1951-01-01 00:00:00, 1952-01-01 00:00:00], dtype='object')
print(times.shift(1, 'YS'))
>>> CFTimeIndex([1951-01-01 00:00:00, 1952-01-01 00:00:00, 1953-01-01 00:00:00], dtype='object')I am using this heavily in my package to shift large CFTimeIndex's many times in succession. It's running fairly slow and we're finding that the bottleneck is most likely in cftime rather than xarray.CFTimeIndex.shift(). Please see @spencerkclark's timing comments toward the bottom of that issue discussion.
It looks like the calculation of dayofwk and dayofyr is the bottleneck here, since it's being computed on initialization of the object (
|
if self.dayofwk < 0: |
|
jd = JulianDayFromDate(self,calendar='365_day') |
|
year,month,day,hour,mn,sec,ms,dayofwk,dayofyr =\ |
|
DateFromJulianDay(jd,return_tuple=True,calendar='365_day') |
|
self.dayofwk = dayofwk |
|
self.dayofyr = dayofyr |
).
@spencerkclark showed that there are huge speedups from commenting that out.
Would it be feasible to make these dynamic attributes? E.g. they are calculated as a property of the object only when requested by the user?
Details:
cftime version: 1.0.4.2
environment: Mac OSX, python 3.7.3
I am moving a discussion from
pangeoover to here: pangeo-data/pangeo#764.xarrayhas a method on theirCFTimeIndexto shift thecftime.E.g.,
I am using this heavily in my package to shift large
CFTimeIndex's many times in succession. It's running fairly slow and we're finding that the bottleneck is most likely incftimerather thanxarray.CFTimeIndex.shift(). Please see @spencerkclark's timing comments toward the bottom of that issue discussion.It looks like the calculation of
dayofwkanddayofyris the bottleneck here, since it's being computed on initialization of the object (cftime/cftime/_cftime.pyx
Lines 1468 to 1473 in 9f7c576
Would it be feasible to make these dynamic attributes? E.g. they are calculated as a property of the object only when requested by the user?
Details:
cftimeversion: 1.0.4.2environment: Mac OSX, python 3.7.3