Extending ORCA blockplotting (w/ tests) #3135

lbdreyer · 2018-08-13T11:33:09Z

This adds some changes to the testing of the 2d blockplotting (and updates some things that fell out from testing it.)

I have focused on tests for the changes to iris/coords.py so far but will add tests for the changes to iris/plot.py once I have them (and once we have decided on what to do about discontiguity checking)

lbdreyer · 2018-08-13T11:37:42Z

lib/iris/tests/unit/plot/test_2d_coords.py

@@ -1,157 +0,0 @@
-# (C) British Crown Copyright 2018, Met Office


I have removed this file as it had mixed unit tests for lots of different functions.
Unit tests should be separated so that each file tests a specific function/class (e.g. a single file for testing iris.plot._get_plot_defn)

I have moved the tests in this file to the appropriate files for almost all of them. Have not yet done it for the plot.py functions but will also push that up to this PR once that is done

Cool, it was on my list of things to do. In my own PR I have shuffled some tests around and added some more for the new utils, but these were always intended to be cleaned up anyway, it's just a bit easier to run tests when they're in the same place.

I guess I can pull in your changes and then move the rest of the tests round accordingly, because none of them are in the right place.

stickler-ci · 2018-08-15T11:03:52Z

lib/iris/coords.py

-                             '2 dimensions.'.format(self.name()))
+            raise ValueError('Invalid operation for {!r}. Not supported for '
+                             'bounds with more than 2 dimensions.'.format(
+                self.name()))


E122 continuation line missing indentation or outdented

stickler-ci · 2018-08-15T11:03:53Z

lib/iris/tests/unit/plot/test__get_plot_defn_custom_coords_picked.py

+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with Iris.  If not, see <http://www.gnu.org/licenses/>.
+"""Unit tests for the `iris.plot._get_plot_defn_custom_coords_picked` function."""


E501 line too long (82 > 79 characters)

stickler-ci · 2018-08-15T11:03:53Z

lib/iris/tests/unit/plot/test__get_plot_defn_custom_coords_picked.py

+    def test_span_check(self):
+        cube = hybrid_height()
+        emsg = 'The given coordinates (sigma, level_height) don\'t span the '
+               '2 data dimensions.'


E999 IndentationError: unexpected indent
E111 indentation is not a multiple of four
E113 unexpected indentation

stickler-ci · 2018-08-15T11:03:53Z

lib/iris/tests/unit/plot/test__get_plot_defn_custom_coords_picked.py

+
+
+if __name__ == "__main__":
+    tests.main()


W292 no newline at end of file

stickler-ci · 2018-08-15T12:32:21Z

lib/iris/tests/integration/plot/test_plot_2d_coords.py

+
+# Run tests in no graphics mode if matplotlib is not available.
+if tests.MPL_AVAILABLE:
+    import matplotlib.pyplot as plt


F401 'matplotlib.pyplot as plt' imported but unused

stickler-ci · 2018-08-15T12:38:42Z

lib/iris/tests/unit/plot/test__get_plot_defn_custom_coords_picked.py

+
+if __name__ == "__main__":
+    tests.main()
+


W391 blank line at end of file

lbdreyer · 2018-08-15T14:03:44Z

@pp-mo @corinnebosley
Could someone look at this?

I have changed:

Docs strings for a couple functions
added support to cubes that may be of shape (2,1) or (1,2) or (1) to the discontiguity check
changes contiguous bounds to check that the bounds are contiguous (before it was assumed)*
Added a check in plot.py that the coordinates have the same number of dimensions (to stop someone doing a 1D coord against a 2D coord
Removed the ability to modify the absolute tolerance when calling pcolormesh (instead use default)
Added unit tests for:
- coord methodscontiguous_bounds, is_contiguous and sanity_check_bounds
- coords._discontiguity_in_2d_bounds
- coords._get_2d_coord_bound_grid
- plot.get_plot_defn
- plot.get_plot_defn_custom_coords_picked
Added integration test for pcolormesh - will also need to add the expected graphic for this

Note I have not written any tests for _check_contiguity_and_bounds or modified how we call that (other than to remove the abs_tol thing) but it does need fixing as currently it isn't right (we're calling it too many times) but this may be covered by what @corinnebosley is doing so I thought I'd leave it.

Doing this has meant that when checking for discontiguities we are using the default absolute tolerance value which is 1e-8 which may be too small (it is too small for our ORCA2 test data).

pp-mo · 2018-08-15T14:59:13Z

it is too small for our ORCA2 test data

I'm worried about the practical usability of this.
If I understand right, between this and the bounds-check-routine that @corinnebosley is working on, this means that :

some ORCA data will generate warnings when plotted, because it has quite large discontiguities at certain points
the new contig-check routine will let you check the bounds with adjustable tolerance, to confirm whether the problem is significant
but the only provided way to avoid the plot warning is to mask out the problem points. Which in fact you don't necessarily want to do.

So, I think it is necessary for the plot routine to either have a much more generous default tolerance, or a way to modify it.

IMHO the only point of warnings is to encourage people to take action to avoid them, so there should always be a way to avoid it, and ideally the warning tells you how : If there isn't a way out, then you are just giving encouragement to ignore them, and may as well not bother !

corinnebosley · 2018-08-16T08:17:28Z

lib/iris/coords.py

-                             '2 dimensions.'.format(self.name()))
+            raise ValueError('Invalid operation for {!r}. Not supported for '
+                             'bounds with more than 2 dimensions.'.format
+                             (self.name()))


I'm not sure about this error message. It seems slightly more confusing than it was previously.

The name is better though. Much clearer.

I think I changed this because I wanted the method _sanity_check_bounds to be able to be used for other things, but the previous error message.
Saying that, I realise I have made a lot of breaking changes in this PR, the above being one of them so I am going to undo that and put the error message back

corinnebosley · 2018-08-16T08:25:17Z

lib/iris/coords.py

        return contiguous

    def contiguous_bounds(self):
        """


Docstring is better (I like things which are concise) but I'm still not taken with the name. 'contiguous_bounds' is just a bit vague.

Unfortunately we don't have much say with changing the name of contiguous_bounds. It was already part of the API so changing it would be a breaking change.

corinnebosley · 2018-08-16T08:28:23Z

lib/iris/coords.py

+            if not self.is_contiguous():
+                raise ValueError('Invalid operation. Bounds of coord {!r} '
+                                 'are not contiguous.'.format(self.name()))
+


Nice catch. Much better.

Unfortunately this is a breaking change so I am going to have to put this back

corinnebosley · 2018-08-16T08:33:18Z

lib/iris/plot.py

+        if not (len(spans[0]) == len(spans[1])):
+            raise ValueError('The coordinate {!r} has {} dimensions and the '
+                             'coordinate {!r} has {} dimensions. Cell-based '
+                             'plotting is only supported for coordinates with '


I'd like to discuss this with you, because I am a little unsure of what cell-based plotting means. Also whether it is clear to users what cell-based plotting is. And whether it matters, I suppose.

corinnebosley · 2018-08-16T08:35:29Z

lib/iris/plot.py

-    Check that the discontinuous bounds occur where the data is masked.
+    Checks that any discontiguity in the bounds of the coordinate only occur
+    where the data is masked.



This is neater and more accurate, but I have made some changes to the name and docstring for this function as well. Probably worth agreeing on changes.

corinnebosley · 2018-08-16T09:13:17Z

lib/iris/tests/unit/coords/test_Coord.py

+    def test_1d_coord_contiguous_atol(self):
+        coord = iris.coords.DimCoord(self.points/1000, bounds=self.bounds/1000)
+        result = coord.is_contiguous(atol=1e-2)
+        self.assertTrue(result)


Might need you to explain this one.

corinnebosley · 2018-08-16T09:14:35Z

lib/iris/tests/unit/coords/test_Coord.py

+    def test_coord_1d_2_bounds(self):
+        coord = iris.coords.DimCoord([0, 1], standard_name='latitude',
+                                     bounds=np.array([[0, 1], [1, 2]]))
+        coord._sanity_check_bounds()


This is just to check that no error is raised is it?

Yep I'll add a comment explaining that

corinnebosley · 2018-08-16T09:19:23Z

lib/iris/tests/unit/coords/test_Coord.py

+                                     bounds=np.array([[0], [1]]))
+        emsg = "Invalid operation for 'latitude', with 1 bounds. Contiguous " \
+               "bounds are only defined for 1D coordinates with 2 bounds."
+        with self.assertRaisesRegexp(ValueError, emsg):


Shouldn't this raise an error about 2d coordinates needing 4 bounds, rather than 1d coordinates needing 2 bounds?

Ah sorry - incorrectly written test! Will change this

corinnebosley · 2018-08-16T09:23:08Z

lib/iris/tests/unit/coords/test_Coord.py

+               "with more than 2 dimensions."
+        with self.assertRaisesRegexp(ValueError, emsg):
+            coord._sanity_check_bounds()
+


Nice work adding all these tests. Really needed them. I'm feeling a lot of technical debt doing this work, and this addresses a good portion of that.

corinnebosley · 2018-08-16T09:36:10Z

lib/iris/tests/unit/coords/test__discontiguity_in_2d_bounds.py

+        self.assertTrue(not diffs_along_x.any())
+        self.assertTrue(not diffs_along_y.any())
+
+    def test_discontiguous_along_x(self):


A short comment about what you are doing to the bounds array here might be useful.

corinnebosley · 2018-08-16T09:47:45Z

lib/iris/tests/unit/coords/test__discontiguity_in_2d_bounds.py

+    def test_contiguous_abs_tol(self):
+        bds = self.discontiguous_bds / 1000
+        x_exp_result = np.array([1, 0]).reshape(2, 1) / 1000
+        y_exp_result = np.array([1, 0]).reshape(1, 2) / 1000


Don't really understand what's happening here.

stickler-ci · 2018-08-23T15:24:35Z

lib/iris/coords.py

+            contiguous = np.allclose(self.bounds[1:, 0],
+                                     self.bounds[:-1, 1],
+                                     rtol=rtol, atol=atol)
+            diffs = np.abs(self.bounds[:-1, 1]- self.bounds[1:, 0])


E225 missing whitespace around operator

stickler-ci · 2018-08-23T15:24:35Z

lib/iris/coords.py

-            elif self.ndim == 2:
-                contiguous, _, _ = _discontiguity_in_2d_bounds(self.bounds,
-                                                               abs_tol=atol)
+            contiguous, _ = _discontiguity_in_bounds(self.bounds, rtol=rtol,


F821 undefined name '_discontiguity_in_bounds'

stickler-ci · 2018-08-23T15:24:36Z

lib/iris/plot.py

-                             ' suitable mesh to give to'
-                             ' Matplotlib'.format(coord.name()))
+            not_masked_at_discontiguity = not_masked_at_discontiguity_along_x \
+                    or not_masked_at_discontiguity_along_y


E127 continuation line over-indented for visual indent

stickler-ci · 2018-08-23T15:24:36Z

lib/iris/plot.py

+                             'contiguous and data is not masked where the '
+                             'discontiguity occurs. Not able to create a '
+                             'suitable mesh to give to Matplotlib'.format(
+                coord.name()))


E122 continuation line missing indentation or outdented

stickler-ci · 2018-08-23T15:24:36Z

lib/iris/plot.py

                                                       kwargs)

-    if arg_func is not None:
+        if arg_func is not None:


E999 IndentationError: unexpected indent
E113 unexpected indentation

stickler-ci · 2018-08-27T21:59:52Z

lib/iris/coords.py

+            def mod360_adjust(compare_axis):
+                bounds = self.bounds.copy()
+
+                if compare_axis = 'x':


E999 SyntaxError: invalid syntax

stickler-ci · 2018-08-27T21:59:52Z

lib/iris/coords.py

+                return diffs_along_axis, contiguous_along_axis
+
+
+            diffs_along_x, match_y0_x1 = mod360_adjust(compare_axis='x')


E303 too many blank lines (2)

stickler-ci · 2018-08-27T21:59:52Z

lib/iris/tests/unit/coords/test_Coord.py

+        err_msg = 'Bounds for 2D coordinates must be 3-dimensional and have ' \
+                  '4 bounds per point'
+        with self.assertRaisesRegexp(ValueError, err_msg):
+            _discontiguity_in_2d_bounds(bds)


F821 undefined name '_discontiguity_in_2d_bounds'

stickler-ci · 2018-08-27T21:59:52Z

lib/iris/tests/unit/coords/test_Coord.py

+        err_msg = 'Bounds for 2D coordinates must be 3-dimensional and have ' \
+                  '4 bounds per point'
+        with self.assertRaisesRegexp(ValueError, err_msg):
+            _discontiguity_in_2d_bounds(bds)


F821 undefined name '_discontiguity_in_2d_bounds'

stickler-ci · 2018-08-27T21:59:52Z

lib/iris/tests/unit/coords/test_Coord.py

+
+    def test_contiguous_both_dirs(self):
+        bds = self.lon_bounds_3by3
+        all_eq, diffs_along_x, diffs_along_y = _discontiguity_in_2d_bounds(bds)


F821 undefined name '_discontiguity_in_2d_bounds'

stickler-ci · 2018-08-27T21:59:53Z

lib/iris/tests/unit/coords/test_Coord.py

+        bds = self.discontiguous_bds / 1000
+        x_exp_result = np.array([1, 0]).reshape(2, 1) / 1000
+        y_exp_result = np.array([1, 0]).reshape(1, 2) / 1000
+        all_eq, diffs_along_x, diffs_along_y = _discontiguity_in_2d_bounds(


F821 undefined name '_discontiguity_in_2d_bounds'

stickler-ci · 2018-08-27T21:59:53Z

lib/iris/tests/unit/coords/test_Coord.py

+        bds = self.discontiguous_bds
+        x_exp_result = np.array([1, 0]).reshape(2, 1)
+        y_exp_result = np.array([1, 0]).reshape(1, 2)
+        all_eq, diffs_along_x, diffs_along_y = _discontiguity_in_2d_bounds(


F821 undefined name '_discontiguity_in_2d_bounds'

stickler-ci · 2018-08-27T21:59:53Z

lib/iris/tests/unit/coords/test_Coord.py

+    def test_contiguous_mod_360(self):
+        bds = np.array([[[170, 180, 180, 170], [-180, -170, -170, -180]],
+                        [[170, 180, 180, 170], [-180, -170, -170, -180]]])
+        all_eq, diffs_along_x, diffs_along_y = _discontiguity_in_2d_bounds(bds)


F821 undefined name '_discontiguity_in_2d_bounds'

stickler-ci · 2018-08-27T21:59:54Z

lib/iris/tests/unit/coords/test_Coord.py

+                        [[170, 180, 180, 170], [-180, -170, -170, -180]]])
+        x_exp_result = np.array([360, 0])
+        y_exp_result = np.array([0, 0])
+        all_eq, diffs_along_x, diffs_along_y = _discontiguity_in_2d_bounds(bds)


F821 undefined name '_discontiguity_in_2d_bounds'

stickler-ci · 2018-08-27T21:59:54Z

lib/iris/tests/unit/coords/test_Coord.py

+
+
+
+class Test__sanity_check_bounds(tests.IrisTest):


E303 too many blank lines (3)

…ce checking when plotting

lbdreyer · 2018-08-28T10:59:48Z

Summary of changes

Discontiguity checking:

Moved _discontiguity_in_2d_bounds function to a method on the coord so it could easily call other coord methods
Rewrote _discontiguity_in_2d_bounds -> _discontiguity_in_bounds so that it supports 1D and 2D coords, and calls np.allclose
coord.is_contiguous now calls coord._discontiguity_in_bounds. 1D coordinates are still handled as before (so this doesn't introduce a breaking change) but the handling has now been moved into _discontiguity_in_bounds

2d coord plotting:

Introduced contiguity_tolerance kwarg to pcolor and pcolormesh
- If it is set, the bounds of both 1D and 2D coords are checked for discontiguities using an absolute toelrance as set
- If it isn't set, 1D coords are not checked, but 2D corods are checked with the defaults
If discontiguities are found, an error is raised

Missing from this PR:

An integration test but that will be easier to include in a separate PR

lbdreyer · 2018-08-28T15:39:35Z

@pp-mo The tests are now passing, do you mind having a look?

pp-mo

Quite a few comments, but all pretty minor.

lib/iris/plot.py

lib/iris/coords.py

lib/iris/tests/unit/coords/test_Coord.py

lbdreyer · 2018-08-31T09:39:16Z

@pp-mo I believe I have addressed all your review comments in the most recent commit.

pp-mo · 2018-08-31T15:35:13Z

@lbdreyer pp-mo I believe I have addressed all your review comments in the most recent commit.

Yes, I'm happy that we are all done here now.
Result 👍

I'll merge, and hopefully all will check out, since #3150 is now also merged.

@corinnebosley

* Fix coord_comparison when coords are the same object * Added what's new items for SciTools#2730 and SciTools#2325. (SciTools#2852) * Added what's new items for SciTools#2730 and SciTools#2325. [skip ci] * Moved the bugfix docs to enhancement for the 3x speedup of area weights. [skip ci] * Add unit test * Add workaround * Add test of current dask behaviour * Ammended cube.var_name documentation. (SciTools#2874) Ammended cube.var_name documentation to refer to it as NetCDF. * Futures netcdf no unlimited (SciTools#2838) * switched boolean operator and altered docstring * raised error on future flag and tweaked functionality * small change to deprecation warning message * Code changes * remake cdl results * More CDL updates (cherry picked from commit 47272b3) * Review actions * Use default Dask scheduler settings (SciTools#2879) Disable dask.set_options call on _lazy_data import. * Fix doctest failing due to change in 'netcdf_promote' future behaviour (SciTools#2883) * Remove unneeded image repo url (SciTools#2885) * Updated 2.0 release notes. (SciTools#2884) * Add iris-grib as a testing dependency; Python 3 included. * Fix requirements for iris_grib>=0.12, assuming it will have eccodes + thus work with Python 3 * Made the iris extensions not install by default. * Reduced the number of processes used by default when testing. * Set the documentation build to use viridis as the default colormap (rather than Jet as we are using matplotlib 1 still). * Change behaviour of 'error' level (SciTools#2881) * Change behaviour of 'error' level * Change Future netcdf_no_unlimited tests * Fix tests testing the wrong thing * Dask processing options docs updates * Added some what's new information about SciTools#2435 and SciTools#2810. * Use the cube dimension order to determine the plot axes. * Small update to experimental-FF whatsnew statement (SciTools#2898) * Move some whatsnew items, remove spurious list * Move nc-no-unlim text to whatsnew * Fill value handling on save notes * Netcdf scalar cube loading bugfix Previously Iris used to simply ignore scalar cubes. * Corrected usage context in description of PointInCell regrid scheme. (SciTools#2735) * Re-worked setup.py to avoid the need for separate/non-standard build commands (SciTools#2891) Re-worked setup.py to avoid the need for separate/non-standard build commands. * Missing data whitepaper (SciTools#2906) * Ensure lon/lat return order is consistent throughout (SciTools#2238) * Updated to v46 of the CF standard-name table. (SciTools#2908) * Updated to v46 of the CF standard-name table. * Updated the area-type table to v6. * Use viridis cmap for doc images (SciTools#2909) * Stickler config (SciTools#2896) * Added a flake8 config. * Added a stickler-ci config. * Handled the recent update to conda-forge for testing, perhaps relating to GEOS update. (SciTools#2912) * Bumped master to the v2.1 stream. (SciTools#2910) * Stickler config (SciTools#2896) * Added a flake8 config. * Added a stickler-ci config. * Improved the way requirements are handled, ensuring that pip gets all the information it needs. (SciTools#2890) Improved the way requirements are handled, ensuring that pip gets all the information it needs. * Further docs refinement for v2.0.0 (SciTools#2916) Some minor documentation tweaks for v2.0.0 * Remove obsolete reference from Userguide. (SciTools#2915) * Reverse 2878 (SciTools#2926) * Remove dask issue workaround; wrap computed MaskedConstant as array * Reduce exception message for loading missing files. (SciTools#2925) * reduce error message * update file not found tests * clearer message * Allow missing value (SciTools#2892) * Allow missing_value as a user set attribute This allows the user to set missing_value as an attribute on a cube while ensuring that it is saved as local attribute in the netcdf file. * Added the what's new and updated the copyright years for SciTools#2892. * Fix some header dates. * Add note on using intersection (SciTools#2931) * CML changes for cf_units v1.2.0 (SciTools#2936) * CML changes for cf-units v1.2.0 * Add whatsnew entry * [DOC]: Update userguide note on intersection (SciTools#2933) * Update userguide note on intersection * Update with concrete boundary examples * Prepare master for v2.0.x branch (SciTools#2934) * Converted the contributions into the whatsnew. * Updated to a dev version string. * Tweaked what's new, bumped version string, and updated the MANIFEST for pypi. * Perserve dtype of source cube with area weighted regridder (SciTools#2203) Perserve dtype of source cube with area weighted regridder & add what's new * Update license headers to handle the merge of a PR from 2017 in 2018 (SciTools#2948) * DOC: lazy data docstring made clearer (SciTools#2947) * DOC: lazy data docstring made clearer * Clarify "view" * Remove gdal from 'all' to 'extensions' depedency group. * Remove gdal from 'all' to 'extensions' depedency group. * DOC: lazy data docstring made clearer (SciTools#2947) * DOC: lazy data docstring made clearer * Clarify "view" * No docs are better than wrong docs on dask * Add a general zenodo link to the readme. (SciTools#2955) * Improved the pip installability of scitools-iris. (SciTools#2956) * Release of iris v2.0.0 🎉 * Merge v2.0.x into master (SciTools#2957) * Tweaked what's new, bumped version string, and updated the MANIFEST for pypi. * Remove gdal from 'all' to 'extensions' depedency group. * DOC: lazy data docstring made clearer (SciTools#2947) * DOC: lazy data docstring made clearer * Clarify "view" * No docs are better than wrong docs on dask * Add a general zenodo link to the readme. (SciTools#2955) * Improved the pip installability of scitools-iris. (SciTools#2956) * Release of iris v2.0.0 🎉 * Move v2.0.x branch into development mode. * Fix the documentation building for latest version of sphinx. (SciTools#2958) * Remove FutureWarning from use of np.issubdtype(obj, np.str) (SciTools#2954) * Add cube co-realisation. (SciTools#2967) * Fix for changed API of cartopy.crs.Stereographic (SciTools#2971) * Fix for changed API of cartopy.crs.Stereographic; fixes tests.test_coordsystem. * License header fix. * remove site.cfg exclude from manifest.in (SciTools#2978) * Save calendar attribute for cube data (SciTools#2984) * Fix bug with aux coords with scalefactor/add_offset (SciTools#2975) Fix bug with aux coords with scalefactor/add_offset * Rename to correct package name * Fix imagehash hex_to_hash (SciTools#2989) * Fix imagehash hex_to_hash * expose hamming distance vector * fix idiff * update imagerepo * fix repo hashes (bjlittle) * fix repo hashed (scitools) * fix test_image_json * test_pearsonr: use compatible cubes for testing compatible cubes (SciTools#2960) Pearsonr test using compatible cubes * Use unittest.mock for Python 3 (SciTools#3003) * reflect new latitude guess_bounds behaviour (SciTools#3006) * Pin Dask to >=0.17.1 (SciTools#2965) * Fill a usability gap in iris.analysis.Trajectory (SciTools#2770) * Fill a usability hole in Trajectory * Remove iris.experimental.um module (SciTools#2781) * Remove iris.experimental.um module * Replace use of sphinx's status_iterator function, which has moved * Lazy unit conversions for cube and coord data. (SciTools#2964) * Gracefully handle long time intervals (SciTools#2354) Gracefully handle long time intervals (months & years) when plotting * Prevent creation of invalid CF variable names. (SciTools#3009) Prevent creation of invalid CF variable names. * Add stash check to name() (SciTools#2930) Try the STASH attribute when checking for possible names (as is already done in the summary method) * Attempts to fix docs-build error reporting. (SciTools#2983) Remove final semicolon to report docs-build error. * Phenomenon translation updates (SciTools#3011) * Netcdftime dep (SciTools#3017) Pin to netcdf4 1.3 or below while we await netdftime updates Correct deprecated pandas use in v0.23 * add 'axes' kwarg to _label when called from contourf * Replace iris.co_realise_cubes with CubeList.realise_data. (SciTools#3013) * Replace iris.co_realise_cubes with CubeList.realise_data. * Avoid exposing lazy-data functions in iris.cube and iris.coord modules. * Tidy lazy_data imports in cube + coords modules. * Amend var_name documentation (SciTools#3020) Amend documentation to be explicit that var_name is a netCDF quantity, not necessarily a CF one. * Updated tests for cf_units 2 (SciTools#3023) Updated tests for cf_units>=2 This change: * Removes the redundant ``iris/tests/test_unit.py`` tests. They are all migrated to cf_units. * Makes use of CALENDAR_GREGORIAN, rather than CALENDAR_STANDARD. These calendars are implemented as aliases in cf_units 2, and the "STANDARD" concept is not preserved/round-tripped. Added a what's new to help reduce possible impact for users. * Use ConfigParser for py>=3.2 (SciTools#3025) * WIP: Unpin mpl (SciTools#3019) * unpin mpl * Add _repr_html_ for Iris cubes (SciTools#2918) Add cube _repr_html_ functionality * Partial collapse of multi-dimensional coordinates (SciTools#3028) Adding support for partial collapse of multi-dimensional coordinates * Tidy up the root of the repository for v2.1 (SciTools#3034) A bunch of miscellaneous changes in preparation for the v2.1 release: * Move CONTRIBUTING.md to a .github/ folder. See also: https://help.github.com/articles/setting-guidelines-for-repository-contributors/ * Updated the .gitignore to reflect what gets produce in day-to-day iris development * Reduced the amount of redundancy in INSTALL. It now points to the requirements/ directory which is what we actually test. * Updated minimum dependencies in requirements (because that is all we test): * Numpy>=1.14 * Matplotlib>=2 * cf_units>=2 * Remove the CODING matrix item. (SciTools#3036) * Add Albers Equal Area Projection and some test as per SciTools#2496 (SciTools#2943) Add Albers Equal Area Projection and NetCDF rules to handle it * Improve the appeal of the landing pages (docs and README) (SciTools#3040) * Update the Iris README markdown to be consistent with other SciTools projects. * Gave the package a description (for pypi). * Minor spruce of the Iris docs layout to include the tagline and the forkme button * Use the correct pypi name for nc-time-axis. (SciTools#3039) * Use cftime instead of netcdftime (SciTools#3016) * use cftime instead of netcdftime * fix the docs too * fix license year * review actions * simplify test_pandas * what's new * remove cf_units check * Fixed the intermittent test in SciTools#3035. (SciTools#3037) * Fixed the intermittent test in SciTools#3035. The problem was that iris.util.format_array was stateful (via np.set_printoptions). * Fix stateful tests (SciTools#26) * Add dask-array as a requirement when pip installing. (SciTools#3033) * Unpin netcdf4 (SciTools#3031) * unpin netcdf4 * adapt for netcdf4 >=1.4 * fix userguide doctests * Metarelate updates in the UM 2 CF mapping (SciTools#3043) * Updated the README, and copied that over to the iris docs homepage. (SciTools#3045) * Added the what's new for iris 2.1 (SciTools#3044) Added the what's new for iris 2.1 * Updated the cf-standard names to v54. (SciTools#3046) Achieved with: $ wget http://cfconventions.org/Data/cf-standard-names/54/src/cf-standard-name-table.xml -O etc/cf-standard-name-table.xml ... --2018-06-02 06:32:17-- http://cfconventions.org/Data/cf-standard-names/54/src/cf-standard-name-table.xml ... * iris.quickplot: Ensure labels and plot use the same axes Allow iris.quickplot to accept an axes kwarg to which title and axes labels can be attached. This is done without using the matplotlib state machine. Closes SciTools#2946 * sticklering * Full-featured Mercator coord system (SciTools#3041) Add ``standard_parallel`` to the Mercator Coord System * Updated the branding and execution of Iris docs (SciTools#3050) * Updated the branding and execution of Iris This includes: * removing the "easy-to-use" statement in Iris' strapline. Let's let users be the judge of that. * improved the styling of the homepage, including re-instating the search bar. * Added a what's new for SciTools#3010. * Added a extlinks section for convenient linking to github issues with :issue:`number` and :pr:`number`. * Added a favicon for the docs. * Improve strapline style, content (SciTools#27) * Further improvements to Iris docs * Improved font (noto-sans) * Removed old CSS * Fixed a few what's new entries * [DOC] Iris Introduction updates (SciTools#3052) Update the introduction of Iris content. * Updated to v2.1.0. (SciTools#3051) * Move into development mode for v2.1.1 * Move into development mode for v2.2.0 * remove first 'and' from tagline to make it read easier * Notebook repr whatsnew update * remove first 'and' from tagline to make it read easier * minor requirements tidy * http -> https for copybutton.js (SciTools#3066) * Explain use of saver='string' in iris.save (SciTools#3062) * Fix reStructuredText warnings in documentation (SciTools#3069) * Add information about comparing STASH objects with strings (SciTools#3072) * Doc: Fallback to no syntax hightlighting without warning. (SciTools#3071) Fixes: iris/docs/iris/src/developers_guide/gitwash/development_workflow.rst:317: WARNING: Could not lex literal_block as "python". Highlighting skipped. http://www.sphinx-doc.org/en/stable/config.html#confval-highlight_language * Quick fix to get tests passing by pinning dask version (SciTools#3086) * ENH: Working zonal mean linear regridding for circular sources or with use of extrapolation (SciTools#3085) * Working zonal mean from circular/extrapolated source to target * MAINT: Refactor of zonal mean testing * MAINT: Documentation changes from review * Avoid pandas deprecation warning. (SciTools#3079) Avoid pandas deprecation warning. This also simplifies code by being explicit about usage of "base" and "values" for numpy and pandas objects, and removes legacy code that may no longer be valid. * Workaround for dask array copy bug (SciTools#3088) * add contact options to README * Add stash mapping changes to v2.1.0 changelog * Pin Dask for avoid 0.18.2 bug with masked arrays. * Tiny fix for dask, Python3 only? * Disable doctests for Python 2. * Ignore warnings and update array printouts. * Better way to disable Python2 doctests. * Support plotting 2D bounded coords * Add gridcell_angles and rotate_grid_vectors to iris.analysis.cartography, with tests: INCOMPLETE WIP * Roughly working, snapshotted with complex test plot code, to be reduced. * Small improvements. * First working quiver+streamplot. * Handle custom coords correctly * some words for docs, WIP probably * suggested tweaks and corrections * Update docs to included 2.2 versions (SciTools#3110) * Provide a test skipper for 2d coords WIP. (SciTools#3099) * Change version number (SciTools#3118) * re-added bounds definition for contiguity check, removed unnecessary test * Ensure Sphinx autodocs for grid_angles routines. * Codestyle fixes. * Whatsnew entries for 2d vector support. * Pin Dask to avoid 0.18.2 bug with masked arrays. (SciTools#3127) * Put orca_util routines in subpackage of iris.test.stock, to get existing iris.tests.unit.plot.test_2d_coords working. * Disable broken misused testcode in test_gridcell_angles. * Ditch test_gridcell_angles, none of it is functional. * Further style fixes. * Skip tests using iris-test-data, for Travis TEST_MINIMAL phases. * Codestyle fix (though this code obsolete anyway). * Fix unused imports. * Made-up test cube replaces use of iris-test-data. * Renamed keyword; improved docstring. * Change 'co' to 'coord' for clarity. * Review changes. * Update pp save rules to handle time means (SciTools#3132) * Vector plots 2 (SciTools#3120) * Small improvements; first sensible tests. * Enhanced testing; better checking and crs awareness in grid_angles routine. * Remove crud from test_gridcell_angles. * Use degree units for everything in _grid_angles. * Make assertArrayAllClose print details when it fails. * Rework and extend testing for gridcell_angles. * Fix assertArrayAllClose; remove debug code from test_gridcell_angles. * Remove obsolete assignments. * Remove obsolete code. * Small comment improvements. * Attempt to clarify docstrings of low-level routines. * More tests, and some functional fixes. * Codestyle fixes. * Review changes + fixes. * Avoid using sample data. * Temporary pin to avoid problems caused by Proj4 v5.1. (SciTools#3150) * Extending ORCA blockplotting (w/ tests) (SciTools#3135) * Tests for rotate_grid_vectors (SciTools#3148) * Tests for rotate_grid_vectors. * Small fix to assertArrayAllClose for values near 0. * Small tweaks. * Fix test method. * Fix 1 test for 'equal_nans' no longer the default. * Review changes. * Remove 'skip_2d' test decorator. (SciTools#3156) * Add integration graphics tests for pcolormesh with 2d coords (SciTools#3149) * Vector plots 4 (SciTools#3158) * Fix contig_ tolerance kwarg bug (SciTools#3162) * Add simple lazy aggregations * Lazy SUM and COUNT, tests improved * Add tests for MAX and MIN * Update tests for STD_DEV * Use NetCDF variable's chunks on load (SciTools#3131) Preserve NetCDF chunks on dataset load * Iteration and less duplication * Attempt to fix Matplotlib issue by resticting to 2.x * Non-test review actions * Make COUNT tests truer unit tests * Convert MAX and MIN tests to truer unit tests * Convert COUNT and SUM agg tests to truer unit tests * Tests for new _lazy_data function * Fix for netCDF4 1.4 behaviour change. (SciTools#3177) * Improve gracefully filling warning (SciTools#3171) * Realise that running the tests is simpler than I thought * Make Gracefully warning only appear when filling is required * Add tests of warnings * Get rid of install changes, they are in SciTools#3170 * Update after review * Add whatsnew entry * Minor clarification to entry via a little more punctuation * Revert nc vars masking fix (temporarily). (SciTools#3178) * fix for pandas column object bug (SciTools#3176) * turn columns object into pandas array, plus test included cml this time * made things the way pandas like it; bambooish * regenerated cml to match output * Update module headers and installation instructions (SciTools#3170) * Update module headers and installation instructions * Realise that running the tests is simpler than I thought * Update install instructions for developers * Update INSTALL to be consistent * Correct some failing tests * Update tests.rst (SciTools#3082) Added one line on how to test. This is mentioned in the Iris Check List in pulls.rst list, but would be good to have it here as well (useful for new contributors like me). * Add link to developer guide. (SciTools#3093) * Combine identical test routines for pcolor + pcolormesh. * Test 'contiguity_tolerance' use in iris.plot.pcolor and pcolormesh. * Unpin dask * Handle printing bounds of long time interval coords (SciTools#3140) * Handle printing time bounds * Whatsnew, whitespace fix for test result * iris.util.reverse on cubes (SciTools#3155) * make cube reversing official * review: test conventions, etc. * review: enable coord specification * add whatsnew * review: AssertRaises --> AssertRaisesRegexp * cube error handling * Improved Aux factory error handling: better message, delivered earlier (SciTools#3182) * Tests for expected behaviour * Raise error if aux factory relies on a coordinate external to cube * Include coordinate name in error message * Don't check whether dependencies that are "None" are in the coords. * Review changes: check cube name in error and variable rename * Truncated variable for flake8 * Correct cube collapse code * Provide info on pp-field indices in the file for structured um loads. (SciTools#2977) * Field locations for structured um loads. * Remove unused import. * Fix test for new implementation. * Added whatsnew. * Review changes. * simplify dev guide * Integration test for iris.plot.contour with 2d coords. (SciTools#3189) * Integration test for iris.plot.contour with 2d coords. * Review changes. * pinned numpy to not 1.15.2 in core requirements * Public contiguity checking (SciTools#3144) :champagne: :1st_place_medal: for sticking with it @corinnebosley * Populate whatsnew 2.2 (SciTools#3192) * started adding entries to whatsnew * more updates to whatsnew * first complete draft of 2.2 whatsnew * some small corrections * made corrections suggested in review * Update version str * Update CF standard name table to v59 * Fix docstring re structured text warnings (SciTools#3073) (SciTools#3197) * updated version string to 2.3.0dev0 * add contact options to README removed gitter from contact list * Add stash mapping changes to v2.1.0 changelog replaced list with link of stash mappings * expanded name of dev group * added an n for better flow * Fix broken link to dask docs (SciTools#3204)

lbdreyer added the Feature: 2D coords label Aug 13, 2018

lbdreyer commented Aug 13, 2018

View reviewed changes

lbdreyer closed this Aug 14, 2018

lbdreyer deleted the test_2d_coords branch August 14, 2018 08:09

lbdreyer restored the test_2d_coords branch August 14, 2018 12:50

lbdreyer reopened this Aug 14, 2018

lbdreyer force-pushed the test_2d_coords branch from 7212091 to 59f5944 Compare August 15, 2018 11:03

stickler-ci reviewed Aug 15, 2018

View reviewed changes

lbdreyer force-pushed the test_2d_coords branch from 59f5944 to 70da4be Compare August 15, 2018 12:31

stickler-ci reviewed Aug 15, 2018

View reviewed changes

Update and add more tests for blockplotting 2d coords

185241d

lbdreyer force-pushed the test_2d_coords branch from 70da4be to 185241d Compare August 15, 2018 12:38

stickler-ci reviewed Aug 15, 2018

View reviewed changes

corinnebosley reviewed Aug 16, 2018

View reviewed changes

stickler-ci reviewed Aug 23, 2018

View reviewed changes

lbdreyer changed the title ~~Tests for ORCA blockplotting~~ Extending ORCA blockplotting (w/ tests) Aug 24, 2018

stickler-ci reviewed Aug 27, 2018

View reviewed changes

Combine 1d and 2d contiguity checking; do special 2d relative toleran…

78a735d

…ce checking when plotting

lbdreyer force-pushed the test_2d_coords branch from d29bb33 to 27325fd Compare August 28, 2018 10:44

Rewrite contiguity check 2d

e5218bd

lbdreyer force-pushed the test_2d_coords branch from 27325fd to e5218bd Compare August 28, 2018 10:55

lbdreyer force-pushed the test_2d_coords branch from 6449b3d to a2e35e1 Compare August 28, 2018 13:58

support for no mask and tests contiguity check

06b5a01

lbdreyer force-pushed the test_2d_coords branch 2 times, most recently from d620f35 to 06b5a01 Compare August 28, 2018 15:01

pp-mo reviewed Aug 29, 2018

View reviewed changes

lbdreyer force-pushed the test_2d_coords branch 2 times, most recently from 02edb34 to ced5d51 Compare August 31, 2018 09:07

Review actions

70beb41

lbdreyer force-pushed the test_2d_coords branch from ced5d51 to 70beb41 Compare August 31, 2018 09:11

This was referenced Aug 31, 2018

Add images for pcolormesh of cube with 2d_coords SciTools/test-iris-imagehash#16

Merged

Add integration graphics tests for pcolormesh with 2d coords #3149

Merged

pp-mo merged commit cb64761 into SciTools:2d_coords Aug 31, 2018

znicholls pushed a commit to znicholls/iris that referenced this pull request Jun 15, 2019

Extending ORCA blockplotting (w/ tests) (SciTools#3135)

c1ef36f

lbdreyer deleted the test_2d_coords branch September 3, 2019 12:50

		@@ -1,157 +0,0 @@
		# (C) British Crown Copyright 2018, Met Office



		if __name__ == "__main__":
		tests.main() No newline at end of file

		return diffs_along_axis, contiguous_along_axis


		diffs_along_x, match_y0_x1 = mod360_adjust(compare_axis='x')




		class Test__sanity_check_bounds(tests.IrisTest):

Extending ORCA blockplotting (w/ tests) #3135

Extending ORCA blockplotting (w/ tests) #3135

Uh oh!

Conversation

lbdreyer commented Aug 13, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lbdreyer commented Aug 15, 2018

Uh oh!

pp-mo commented Aug 15, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corinnebosley Aug 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lbdreyer Aug 23, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lbdreyer commented Aug 13, 2018 •

edited

Loading

pp-mo commented Aug 15, 2018 •

edited

Loading

corinnebosley Aug 16, 2018 •

edited

Loading

lbdreyer Aug 23, 2018 •

edited

Loading

lbdreyer commented Aug 28, 2018 •

edited

Loading

pp-mo left a comment •

edited

Loading