Repository navigation
Numpy 1.18 from conda ignores __array_function__ override on Windows #15378
Description
Reproducing code example:
This code snippet was provided by @mhvk in astropy/astropy#9871 (comment)
import numpy as np
class A:
def __array_function__(self, *args, **kwargs):
return 'Ha!'
np.fft.fft(A())
# 'Ha!'Error message:
Using Numpy 1.18.1 installed from the defaults channel from conda under a clean Python 3.7 environment on Windows:
~\Miniconda3\envs\test-conda-numpy\lib\site-packages\mkl_fft\_numpy_fft.py in ff
t(a, n, axis, norm)
158 """
159 x = _float_utils.__downcast_float128_array(a)
--> 160 output = mkl_fft.fft(x, n, axis)
161 if _unitary(norm):
162 output *= 1 / sqrt(output.shape[axis])
mkl_fft\_pydfti.pyx in mkl_fft._pydfti.fft()
mkl_fft\_pydfti.pyx in mkl_fft._pydfti._fft1d_impl()
mkl_fft\_pydfti.pyx in mkl_fft._pydfti.__process_arguments()
ValueError: Invalid axis (-1) specified.
For more information, please see astropy/astropy#9871
Numpy/Python version information:
- Windows 10
- Miniconda
- Python 3.7
- Numpy 1.18.1
Activity
rgommers commentedon Jan 21, 2020
This is a known issue, since
mkl-fftmonkeypatchesnumpy.fft.Not much to be done here except avoiding installing
mkl-fftif you rely on__array_function__.Also note that this monkeypatching predates the introdunction of
__array_function__, and we've discussed with the MKL authors switching away from it. Pyfftw monkeypatched in the same way back then (and probably still does). The consensus at that point was that we need a backend system fornumpy.fft. Which is something that_array_function__is not.rgommers commentedon Jan 21, 2020
This is also exactly why a backend system is on the NumPy roadmap: https://numpy.org/neps/roadmap.html#other-functionality
rgommers commentedon Jan 21, 2020
Actually there's one difference: Pyfftw does the monkeypatching on import: https://github.com/hgomersall/pyFFTW/blob/master/pyfftw/interfaces/numpy_fft.py
mkl-fftin this case, according to the report, overwrites at install time. That's something Anaconda has said they'd get rid of.pllim commentedon Jan 21, 2020
FWIW when we pinned to Numpy 1.17 in
conda, the error went away.And for the record, this is not blocking anything on Astropy, as we have workarounds such as:
conda-forge, which is still 1.17defaultschannelrgommers commentedon Jan 21, 2020
Hmm, then Anaconda undid the fixes of this "Intel patches" stuff probably. Unclear if on purpose or by accident. https://github.com/ContinuumIO/anaconda-issues/issues is the right place to complain about that. If you want to do that, that'd be very welcome. Otherwise I can do it tomorrow.
pllim commentedon Jan 21, 2020
Okay, also reported to ContinuumIO/anaconda-issues#11568
mhvk commentedon Jan 21, 2020
Thanks all! It just goes to show that monkey-patching a distribution is really bad form, wasting time that could be better spend solving real problems. With the bottom line that we have now changed all our documentation to suggesting to just use pip instead of conda, which surely was not the hope.
Anyway, the backend option would be one very nice way to solve this.
rgommers commentedon Jan 22, 2020
You could also recommend
conda-forge, which is anyway starting to get the more attractive alternative. Given conda's performance problems, the full Anaconda install should be for beginners only - and once you use Miniconda, may as well go all in on conda-forge.rgommers commentedon Jan 22, 2020
There'll be new packages in
defaultswith the offendingfft(andrandom) patches removed:)pllim commentedon Jan 22, 2020
ContinuumIO/anaconda-issues#11568 (comment)