BUG: Hang on Windows in scikit-learn with 1.13rc1 and 1.14.dev (maybe due to OpenBLAS 0.3.26 update?) #20294
Description
Describe your issue.
This was originally seen in MNE-Python CI roughly a week ago (March 13) using the scipy developement wheel (scientific-python-nightly-wheel) and reported in scikit-learn see scikit-learn/scikit-learn#28625 for more details.
This can also be reproduced with scipy 1.13rc1. Could it be due to updating OpenBLAS to 0.3.26?
The scikit-learn code does nested parallelism, OpenBLAS within OpenMP. Setting OMP_NUM_THREADS=1 or OPENBLAS_NUM_THREADS=1 avoids the hang. A similar work-around can be used with threadpoolctl.
Insights or suggestions more than welcome!
Reproducing Code Example
from sklearn.metrics._pairwise_distances_reduction import ArgKmin
import numpy as np
import threadpoolctl
# Commenting any of these two lines avoids the hang
# threadpoolctl.threadpool_limits(limits=1, user_api='blas')
# threadpoolctl.threadpool_limits(limits=1, user_api='openmp')
X = np.zeros((20, 14000))
ArgKmin.compute(X=X, Y=X, k=10, metric='euclidean')Error message
Hang, i.e. the script never completes. With one of the work-around it completes in less than one second.
SciPy/NumPy/Python version and system information
1.13.0rc1 1.26.4 sys.version_info(major=3, minor=12, micro=2, releaselevel='final', serial=0)
Build Dependencies:
blas:
detection method: pkgconfig
found: true
include directory: /c/opt/64/include
lib directory: /c/opt/64/lib
name: openblas
openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= ZEN MAX_THREADS=24
pc file directory: c:/opt/64/lib/pkgconfig
version: 0.3.26
lapack:
detection method: pkgconfig
found: true
include directory: /c/opt/64/include
lib directory: /c/opt/64/lib
name: openblas
openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= ZEN MAX_THREADS=24
pc file directory: c:/opt/64/lib/pkgconfig
version: 0.3.26
pybind11:
detection method: config-tool
include directory: unknown
name: pybind11
version: 2.11.1
Compilers:
c:
commands: cc
linker: ld.bfd
name: gcc
version: 10.3.0
c++:
commands: c++
linker: ld.bfd
name: gcc
version: 10.3.0
cython:
commands: cython
linker: cython
name: cython
version: 3.0.9
fortran:
commands: gfortran
linker: ld.bfd
name: gcc
version: 10.3.0
pythran:
include directory: C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-t0bobn5y\overlay\Lib\site-packages/pythr
an
version: 0.15.0
Machine Information:
build:
cpu: x86_64
endian: little
family: x86_64
system: windows
cross-compiled: false
host:
cpu: x86_64
endian: little
family: x86_64
system: windows
Python Information:
path: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-bxqwrdgx\cp312-win_amd64\build\venv\Scripts\python.exe
version: '3.12'