@hdyson Is the plan to replace the existing FF loader, or complement it?

This functionality will still be distinct from "ordinary" loading (iris.load).
The only immediate desire is to provide a proper 'mainstream' form with a supported API, as internal uses (StaGE) are now requiring it.

There is an important technical reason for keeping it separate, because the existing structured load can not cope with arbitrary un-structured input files.
However, I have previously speculated that we might eventually remove those restrictions and use "structured" loading wherever possible ...
Your usecase demonstrates how that is not always desirable, which is a valuable observation.

This implies that any future integrated scheme may need to make special allowance for callbacks.
On the other hand, whatever the future API, it should still be possible to replicate the existing mechanism:

PP and Fieldsfile loading is essentially a combination of iris.load_raw and CubeList.merge ...
... so you can simply do that explicitly.
This is explained at http://scitools.org.uk/iris/docs/latest/whitepapers/um_files_loading.html?highlight=load_raw#overview-of-loading-process
I'm very aware that this relationship between loading and merging is currently not clearly explained in the UserGuide where, arguably, it should be : However, there is a problem doing that because the process is fileformat-dependent : while grib loading works the same way, other formats (e.g. netcdf) don't have separate 2d 'fields', so they obviously do not.

Make structured FF loading public #2083

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions