-
Notifications
You must be signed in to change notification settings - Fork 44
Closed
Description
* Issue 1: Data are different. It is accurate to 7 digits, but not to 8. Could be similar to what's [discussed above](https://github.com/ESMValGroup/ESMValCore/pull/685#issuecomment-698253145). Changing the statistic from 'mean' to 'median' as suggested by Mattia solves the issue on the data.
I also found out where this difference comes from: in the current master branch, the data is passed over from the original cubes into a new array here:
| new_arr = np.ma.empty([len(cubes)] + list(new_shape[1:])) |
The incoming data is float32, whereas the new data type is float64. Consequently, the statistics are calculated on the float64 data, introducing an unrealistically high precision. Later on the dtype is converted again to float32, but at that point, the damage is already done.
I've created a new PR to correct this change and to update the reference data. That way we can confirm that this PR doesn't break the preprocessor.
Originally posted by @Peter9192 in #685 (comment)
Metadata
Metadata
Assignees
Labels
No labels