My feeling is that rather than using the 'sanity checker' to handle exceptions it's probably best just to flag issues that can then be investigated. I also agree that as a first run checking the metadata on each file is complete and accurate would be a good start (we would hope that PCMDI would do this, but that's unlikely).

Physical-consistency checks would also be helpful, but we'd need to consider each variable separately. Some variables lend themselves to 'hard limits' e.g. salinity and precip can never be less than zero (although they are in some CMIP5 models!); others are less obvious (what should the maximum allowable be temperature be?)

I would add a test of the time coords on all merged lists of variables across a range of files. Time arrays should be consistent across all files within the same experiment, monotonic, have no gaps, and no 'overlaps'. In my experience these types of errors have been the biggest and most persistent headache.

There are some simple 'experiment' tests based on global energy balance that both Didier and I have found can highlight some sneaky issues; these would not be applied to all files of course, but are a useful model diagnostic.

Will

From: didiermonselesan <notifications@github.com mailto:notifications@github.com>
Reply-To: CWSL/cwsl-mas <reply@reply.github.com mailto:reply@reply.github.com>
Date: Sunday, 7 June 2015 9:06 AM
To: CWSL/cwsl-mas <cwsl-mas@noreply.github.com mailto:cwsl-mas@noreply.github.com>
Cc: Will Hobbs <will.hobbs@utas.edu.au mailto:will.hobbs@utas.edu.au>
Subject: Re: [cwsl-mas] Sanity checker (#42)

Tim,

Agreed. The first thing would be to log the failures and the reasons why the workflow fails on particular files.
One of the first steps for CMIP or any data files of the same kind would be to check individual files (as produced by the modelling group) to make sure they comply with the a priori agreed 'conventions' regarding, for examples

naming of the file
naming of the file together with meta data (especially inheritance, i.e. parent experiment, time correspondence, etc ...)
data itself, correct missing/fill values, data range given unit given variable, monotonicity and gaps given time frequency and bounds, missing coordinates and coordinates bounds

Then possibly moving to more 'subjective' checks

Physically relevant checks (e.g. daily and seasonal cycles

Note that some packages do perform conventions checking either for single files or when apply across files.

Cheers, Didier

From: Tim Bedin <notifications@github.com mailto:notifications@github.com mailto:notifications@github.com>
Reply-To: CWSL/cwsl-mas <reply@reply.github.com mailto:reply@reply.github.com mailto:reply@reply.github.com>
Date: Friday, 5 June 2015 4:26 pm
To: CWSL/cwsl-mas <cwsl-mas@noreply.github.com mailto:cwsl-mas@noreply.github.com mailto:cwsl-mas@noreply.github.com>
Cc: Didier Monselesan <Didier.Monselesan@csiro.au mailto:Didier.Monselesan@csiro.au mailto:Didier.Monselesan@csiro.au>
Subject: Re: [cwsl-mas] Sanity checker (#42)

Also related to this (and a wider issue than just the plugin) is to work out what to do when the sanity checker fails - how do we deal with 'faulty' data in the archive at NCI?

Reply to this email directly or view it on GitHubhttps://github.com//issues/42#issuecomment-109175388.

Reply to this email directly or view it on GitHubhttps://github.com//issues/42#issuecomment-109656913.

University of Tasmania Electronic Communications Policy (December, 2014).
This email is confidential, and is for the intended recipient only. Access, disclosure, copying, distribution, or reliance on any of it by anyone outside the intended recipient organisation is prohibited and may be a criminal offence. Please delete if obtained in error and email confirmation to the sender. The views expressed in this email are not necessarily the views of the University of Tasmania, unless clearly intended otherwise.

Sanity checker #42

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions