Skip to content

ndimage.morphology functions are broken with numpy 1.13.0 #7493

Closed

Description

@soupault

Reproducing code example:

import numpy as np
import scipy
import scipy.ndimage as ndi

a = np.array([[True, False],[False, True]])
ndi.morphology.white_tophat(a, footprint=ndi.generate_binary_structure(2,1))

Error message:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-10525a989fdd> in <module>()
----> 1 ndi.morphology.white_tophat(a, footprint=ndi.generate_binary_structure(2,1))

/usr/lib/python3.6/site-packages/scipy/ndimage/morphology.py in white_tophat(input, size, footprint, structure, output, mode, cval, origin)
   1695         tmp = grey_dilation(tmp, size, footprint, structure, None, mode,
   1696                             cval, origin)
-> 1697         return input - tmp
   1698 
   1699 

TypeError: numpy boolean subtract, the `-` operator, is deprecated, use the bitwise_xor, the `^` operator, or the logical_xor function instead.

Scipy/Numpy/Python version information:

import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
0.19.0 1.13.0 sys.version_info(major=3, minor=6, micro=1, releaselevel='final', serial=0)

References

numpy/numpy#9251 , scikit-image/scikit-image#2681

Activity

  1. added
    defectA clear bug or issue that prevents SciPy from being installed or used as expected
    on Jun 15, 2017
  2. rgommers commented on Jun 15, 2017

    @rgommers
    Member

    Hmm, that's bad. Given that most morphology operations return boolean arrays and that applying multiple morphology operations in a row is a sensible thing to do, I think this requires fixing reasonably fast (possibly by reverting the numpy changes).

    It's a bit surprising that no one caught this before the 1.13.0 numpy release.

  3. matthew-brett commented on Jun 15, 2017

    @matthew-brett
    Contributor

    Ouch - this is my fault - scipy is using the old Numpy pre-release wheelhouse, which is no longer being updated - WIP fix here: #7494

  4. rgommers commented on Jun 15, 2017

    @rgommers
    Member

    Ouch - this is my fault - scipy is using the old Numpy pre-release wheelhouse, which is no longer being updated - WIP fix here: #7494

    Not your fault at all - our test suite isn't failing even after the 1.13.0 release, so our test coverage is NOK. Thanks for the wheelhouse fix though:)

  5. matthew-brett commented on Jun 15, 2017

    @matthew-brett
    Contributor

    Yes, I think what's happening is that we are picking up the travis-ci virtualenv numpy, because we aren't using the --upgrade flag to pip, for the standard install. In my PR, I have added a switch to our own virtualenv in the travis build, to isolate us from Travis-ci's packages.

  6. charris commented on Jun 15, 2017

    @charris
    Member

    Reverting the error for NumPy 1.13 is looking like the right thing to do.

  7. matthew-brett commented on Jun 16, 2017

    @matthew-brett
    Contributor

    Hmm - I think we're testing correctly now with #7494 - although the full tests are timing out. There's no error on the tests that have run though - are we testing this properly.

  8. rgommers commented on Jun 16, 2017

    @rgommers
    Member

    No, that's what I was saying before - there's no test coverage for boolean inputs to these functions

  9. matthew-brett commented on Jun 16, 2017

    @matthew-brett
    Contributor

    Oops :)

  10. soupault commented on Nov 19, 2017

    @soupault
    ContributorAuthor

    Does not happen on numpy=1.13.3, scipy=1.0.0. Should we consider closing the issue?

  11. soupault commented on Jan 19, 2018

    @soupault
    ContributorAuthor

    Any progress on this issue?
    We have started facing the similar issues in scikit-image again. See https://travis-ci.org/scikit-image/scikit-image/jobs/330587301#L4313, scikit-image/scikit-image#2944. I assume that update to numpy=1.14 is a trigger.

    I'd be glad to help if someone could update me on the state of affairs.

  12. modified the milestones: 1.1.0, 1.0.1 on Jan 19, 2018
  13. rgommers commented on Jan 19, 2018

    @rgommers
    Member

    Thanks @soupault, help would be quite useful here. I think the summary is:

    • This was fixed by a revert of the TypeError change in numpy 1.13.1,
    • We then should have fixed this asap, but didn't,
    • The numpy change was again introduced in 1.14.0, and now we shouldn't be reverting it
    • Instead, let's fix this in ndimage quickly and put it into a 1.0.1 release - I'm happy to do that release soon
    • For fixing this, we need first unit tests (similar to the example at the top of this issue) for all affected morphology functions that pass with 1.13.3 but are now failing with 1.14.0. Then boolean arrays need special-casing with one of np.bitwise_xor, the ^ operator, or np.logical_xor (whichever one matches the old behavior, assuming that was sensible).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    defectA clear bug or issue that prevents SciPy from being installed or used as expectedscipy.ndimage

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions