cc @dkillick in case you have time to review

Thanks!

Travis failed on

from .ma import getmaskarray
E   ImportError: dask.array.ma requires numpy >= 1.11.2

The Python 3.5 build uses numpy 1.11.1 whereas the other build, e.g. 3.6 uses numpy 1.14.1

Travis failed

It did... I'm thinking about that right now - I'll get back to you about that soon!

So the failure is being caused by this import of dask.array.ma. The import causes the NumPy version difference to become a problem, which it wasn't before because dask.array.ma isn't imported anywhere else in the codebase.

I'm not quite sure what to do about this. The obvious easy fix is to increase the NumPy version in the failing test, but I'm not sure whether that version of NumPy was chosen for a good, but not immediately obvious, reason. Otherwise we need to remove the problem import line, but that too has problems: do we duplicate the getmaskarray functionality outside of dask.array.ma? Or, like NumPy, do we duplicate the average function to some extent in the masked array module?

@jcrist do you have any thoughts on this?

Perhaps we guard the import on this line?

    wgt = wgt * (~getmaskarray(a))

if numpy.__version__ > ...;
    from . import ma
    wgt = wgt * ~ma.getmaskarray(a)
...

If the Numpy version number is less than that version then presumably the array isn't a dask masked array anyway, so this should be safe?

I think that code duplicate is not a good idea.

In that case, I think pragmatic: you can set the minor numpy version to 1.11.2 here https://github.com/dask/dask/blob/master/continuous_integration/travis/install.sh#L68 instead of 1.11 The latest digit is for bug fixes, this change should have a tiny impact.

For the other Python versions: 2.7, 3.6 and 3.7, the version of numpy is > 1.11.2. We can do the same for py35. I think we can try a numpy >= 1.11.2 in the travis build and let's see what's happened.

If the minor version of numpy is 1.11 for some good reasons and we don't want to upgrade to the minor version 1.11.2, I'll update the PR with the suggestion of @mrocklin

I think that code duplicate is not a good idea.

I agree - the only reason I suggested it at all is because that's what NumPy does, I think (ma has a separate average function).

If the numpy version is updated, we also have to update the setup.py here https://github.com/dask/dask/blob/master/setup.py#L11

This change implies removing the support for numpy < 1.11.2. I'm going to create a py3.5 env with this numpy version constraint and launch pytest. I hope there is a well-fitted versions combination between all dependencies.

All tests passed with:

• python 3.5
• numpy 1.11.3
• pandas 0.19.2

Curious. For unknown reasons, some tests where there are some da.random.random calls failed. Or from sparse.COO.from_numpy functions. I really don't know.

I don't personally know why the minimum version is set the way it is. @jakirkham might have thoughts.

This was probably just easy to do at the time. If bumping the patch version helps, I'd recommend doing that.

Thanks for your suggestion @jakirkham ! I'll update it this week.

• I moved the test dedicated to the average computation with masked array from test_routines.py to test_ma.py
• The function da.average calls the _average wrapper with is_masked=False
• The function da.ma.average calls the same wrapper with is_masked=True

I remove two commits from my branch about the numpy version and the authors.md file which was removed.

@jcrist is there anything left here that you think needs to be handled?

Nope, looks good to me. Thanks!