Skip to content

Commit 4bac25a

Browse files
authored
Merge a49fccb into fb039ab
2 parents fb039ab + a49fccb commit 4bac25a

File tree

8 files changed

+41
-671
lines changed

8 files changed

+41
-671
lines changed

docs/src/further_topics/missing_data_handling.rst

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ On load, any fill-value or missing data value defined in the loaded dataset
2222
should be used as the ``fill_value`` of the NumPy masked array data attribute of the
2323
:class:`~iris.cube.Cube`. This will only appear when the cube's data is realised.
2424

25+
.. _missing_data_saving:
2526

2627
Saving
2728
------
@@ -37,7 +38,8 @@ For example::
3738
.. note::
3839
Not all savers accept the ``fill_value`` keyword argument.
3940

40-
Iris will check for and issue warnings of fill-value 'collisions'.
41+
Iris will check for and issue warnings of fill-value 'collisions' (exception:
42+
**NetCDF**, see the heading below).
4143
This basically means that whenever there are unmasked values that would read back
4244
as masked, we issue a warning and suggest a workaround.
4345

@@ -51,6 +53,8 @@ This will occur in the following cases:
5153
NetCDF
5254
~~~~~~
5355

56+
:term:`NetCDF Format`
57+
5458
NetCDF is a special case, because all ordinary variable data is "potentially masked",
5559
owing to the use of default fill values. The default fill-value used depends on the type
5660
of the variable data.
@@ -64,6 +68,16 @@ The exceptions to this are:
6468
* Small integers create problems by *not* having the exemption applied to byte data.
6569
Thus, in principle, ``int32`` data cannot use the full range of 2**16 valid values.
6670

71+
Warnings are not issued for NetCDF fill value collisions. Increasingly large
72+
and complex parallel I/O operations unfortunately made this feature
73+
un-maintainable and it was retired in Iris 3.9 (:pull:`5833`).
74+
75+
If you need to know about collisions then you can perform your own checks ahead
76+
of saving. Such operations be run lazily (:term:`Lazy Data`). Here is an
77+
example::
78+
79+
>>> default_fill = netCDF4.default_fillvals[my_cube.dtype.str[1:]]
80+
>>> fill_present = (my_cube.lazy_data() == default_fill).any().compute()
6781

6882
Merging
6983
-------

docs/src/whatsnew/latest.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,8 @@ This document explains the changes made to Iris for this release
4545
💣 Incompatible Changes
4646
=======================
4747

48-
#. N/A
48+
#. Warnings are no longer produced for fill value 'collisions' in NetCDF
49+
saving. :ref:`Read more <missing_data_saving>`. (:pull:`5833`)
4950

5051

5152
🚀 Performance Enhancements

0 commit comments

Comments
 (0)