-
Notifications
You must be signed in to change notification settings - Fork 300
Closed
Milestone
Description
@bjlittle Would it be possible to make regridding in iris lazy? It seems relatively straightforward, because it is an operation that acts only to the horizontal dimensions of a cube and the dask chunks are typically along the time dimension, at least for the files in CMIP, so it could be done one chunk at a time.
Here are some use cases that make this a really desirable feature for the ESMValTool community:
- One of the most memory consuming operations we have in ESMValTool is computing multi-model statistics.
irissupports computing various statistics in a lazy way, but we cannot benefit from this feature, because models typically come on different grids, so we need to regrid the cubes first. Because regridding is not lazy, we can only compute multi-model statistics for a small number of years. - When working with larger datasets, e.g. ERA5 data at an hourly resolution, regridding very quickly runs out of memory.
At the moment regridding is not lazy in iris, data appears to be realized e.g. here:
iris/lib/iris/analysis/_regrid.py
Lines 1044 to 1045 in f5feb28
| data = self._regrid( | |
| src.data, |
bascrezee