Skip to content

python3Packages.numpy: 1.21.5 -> 1.23.1#156578

Closed
mweinelt wants to merge 1 commit intoNixOS:stagingfrom
mweinelt:numpy
Closed

python3Packages.numpy: 1.21.5 -> 1.23.1#156578
mweinelt wants to merge 1 commit intoNixOS:stagingfrom
mweinelt:numpy

Conversation

@mweinelt
Copy link
Member

@mweinelt mweinelt commented Jan 24, 2022 •
edited
Loading

Motivation for this change

numpy updates
https://github.com/numpy/numpy/releases/tag/v1.22.0
https://github.com/numpy/numpy/releases/tag/v1.22.1
https://github.com/numpy/numpy/releases/tag/v1.22.2
numpy/numpy#20886 to fix scipy/scipy#15414

scipy updates
https://scipy.github.io/devdocs/release.1.8.0.html

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Jan 24, 2022
@mweinelt
Copy link
Member Author

mweinelt commented Jan 24, 2022 •
edited
Loading

The scikit-learn tests are now crashing for me.

@ofborg build python3Packages.scikit-learn

[gw69] [100%] PASSED utils/tests/test_validation.py::test_num_features_errors_1d_containers[list-float] Fatal Python error: Segmentation fault

Current thread 0x00007ffff77dc740 (most recent call first):
<no Python frame>
Fatal Python error: Segmentation fault

Current thread 0x00007ffff77dc740 (most recent call first):
<no Python frame>


=================================== FAILURES ===================================
____________________ decomposition/tests/test_kernel_pca.py ____________________
[gw12] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw12' crashed while running 'decomposition/tests/test_kernel_pca.py::test_32_64_decomposition_shape'
____________________ decomposition/tests/test_kernel_pca.py ____________________
[gw11] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw11' crashed while running 'decomposition/tests/test_kernel_pca.py::test_kernel_pca_raise_not_fitted_error'
____________________ decomposition/tests/test_kernel_pca.py ____________________
[gw44] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw44' crashed while running 'decomposition/tests/test_kernel_pca.py::test_kernel_pca_consistent_transform'
_____________________________ tests/test_common.py _____________________________
[gw62] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw62' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_estimators_fit_returns_self(readonly_memmap=True)]'
_____________________________ tests/test_common.py _____________________________
[gw31] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw31' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_estimators_dtypes]'
_____________________________ tests/test_common.py _____________________________
[gw16] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw16' crashed while running 'tests/test_common.py::test_check_n_features_in_after_fitting[LarsCV()]'
_________________________ utils/tests/test_extmath.py __________________________
[gw41] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw41' crashed while running 'utils/tests/test_extmath.py::test_randomized_eigsh_reconst_low_rank[10-7]'
_____________________________ tests/test_common.py _____________________________
[gw47] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw47' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_estimators_nan_inf]'
_____________________________ tests/test_common.py _____________________________
[gw34] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw34' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_dtype_object]'
_____________________________ tests/test_common.py _____________________________
[gw65] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw65' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_transformer_preserve_dtypes]'
_____________________________ tests/test_common.py _____________________________
[gw54] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw54' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_transformer_general]'
_____________________________ tests/test_common.py _____________________________
[gw0] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw0' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_estimators_pickle]'
_____________________________ tests/test_common.py _____________________________
[gw18] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw18' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_estimator_sparse_data]'
_____________________________ tests/test_common.py _____________________________
[gw45] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw45' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_pipeline_consistency]'
_____________________________ tests/test_common.py _____________________________
[gw49] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw49' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_fit_check_is_fitted]'
_____________________________ tests/test_common.py _____________________________
[gw58] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw58' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_fit_idempotent]'
_____________________________ tests/test_common.py _____________________________
[gw46] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw46' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_fit_score_takes_y]'
_____________________________ tests/test_common.py _____________________________
[gw24] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw24' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_transformer_general(readonly_memmap=True)]'
_____________________________ tests/test_common.py _____________________________
[gw56] linux -- Python 3.9.10 /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/bin/python3.9
worker 'gw56' crashed while running 'tests/test_common.py::test_estimators[KernelPCA()-check_transformer_data_not_an_array]'

@FRidh
Copy link
Member

FRidh commented Jan 24, 2022

Probably best to keep for a new python-unstable iteration.

@mweinelt
Copy link
Member Author

mweinelt commented Jan 24, 2022

Wanted ofBorg to check if the error is reproducible, happy to keep this a draft for now.

@mweinelt mweinelt marked this pull request as draft January 24, 2022 19:24
@Scriptkiddi
Copy link
Contributor

Scriptkiddi commented Feb 21, 2022

hey is there any update on this?

@mweinelt
Copy link
Member Author

mweinelt commented Feb 21, 2022

Doing another bump shortly

@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 1001-2500 This PR causes many rebuilds on Darwin and should most likely target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 2501-5000 This PR causes many rebuilds on Linux and should target the staging branches. labels Feb 21, 2022
@mweinelt
Copy link
Member Author

mweinelt commented Feb 21, 2022

@ofborg build python3Packages.scikit-learn

@mweinelt
Copy link
Member Author

mweinelt commented Feb 21, 2022 

=========================== short test summary info ============================
FAILED decomposition/tests/test_kernel_pca.py::test_kernel_pca_consistent_transform
FAILED decomposition/tests/test_kernel_pca.py::test_kernel_pca_raise_not_fitted_error
FAILED decomposition/tests/test_kernel_pca.py::test_32_64_decomposition_shape
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_pipeline_consistency]
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_estimators_dtypes]
FAILED tests/test_common.py::test_check_n_features_in_after_fitting[KernelPCA()]
FAILED utils/tests/test_extmath.py::test_randomized_eigsh_reconst_low_rank[10-7]
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_estimators_pickle]
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_transformer_data_not_an_array]
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_estimator_sparse_data]
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_estimators_overwrite_params]
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_fit_idempotent]
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_estimators_fit_returns_self(readonly_memmap=True)]
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_estimators_nan_inf]
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_estimators_fit_returns_self]
FAILED utils/tests/test_extmath.py::test_randomized_eigsh_reconst_low_rank[100-80]
FAILED tests/test_common.py::test_estimators[KernelPCA()-check_transformer_general]
= 17 failed, 20108 passed, 1588 skipped, 57 xfailed, 38 xpassed, 6836 warnings in 136.58s (0:02:16) =

@mweinelt mweinelt mentioned this pull request Mar 3, 2022
Merged
@risicle
Copy link
Contributor

risicle commented Mar 4, 2022

It's looking like this is some sort of heap corruption. Which is painful to debug because it only throws errors when things are deallocated, long after the culprit has finished executing.

@risicle
Copy link
Contributor

risicle commented Mar 6, 2022 •
edited
Loading 

88f988ae6226ffb5e3c9ae41a2d0cf9fd982109a is the first bad commit
commit 88f988ae6226ffb5e3c9ae41a2d0cf9fd982109a
Author: Pearu Peterson <[email protected]>
Date:   Fri Sep 3 21:56:18 2021 +0300

    Fix a bug in dependencies. Implement support for relational operations.

 numpy/f2py/crackfortran.py        |   7 +-
 numpy/f2py/symbolic.py            | 141 +++++++++++++++++++++++++++++++-------
 numpy/f2py/tests/test_symbolic.py |  38 +++++++++-
 3 files changed, 157 insertions(+), 29 deletions(-)

numpy/numpy@88f988a

numpy/numpy#19805

@risicle
Copy link
Contributor

risicle commented Mar 6, 2022 •
edited
Loading

The suggestion is that the fix for this should be numpy/numpy@258c3ab, but this is present in 1.22.2 already.

Edit: of course I say a fix for "this", but that should be a fix for "an issue caused by that PR". Our issue could be a separate problem with that PR.

@risicle
Copy link
Contributor

risicle commented Mar 7, 2022 •
edited
Loading

Valgrind revealed more in the end:

==401176== Invalid write of size 4
==401176==    at 0x174EF025: dlar1v_ (in /nix/store/413dlxvbzrsfaakf00ffsgw7f1qapx3f-lapack-3/lib/liblapack.so.3)
==401176==    by 0x174F5ED8: dlarrv_ (in /nix/store/413dlxvbzrsfaakf00ffsgw7f1qapx3f-lapack-3/lib/liblapack.so.3)
==401176==    by 0x1756F766: dstemr_ (in /nix/store/413dlxvbzrsfaakf00ffsgw7f1qapx3f-lapack-3/lib/liblapack.so.3)
==401176==    by 0x1752F5DE: dsyevr_ (in /nix/store/413dlxvbzrsfaakf00ffsgw7f1qapx3f-lapack-3/lib/liblapack.so.3)
==401176==    by 0x273B45E0: f2py_rout__flapack_dsyevr (in /nix/store/gmcby9wc7c5agdilcqq8ngzvn0psz2gy-python3.9-scipy-1.8.0/lib/python3.9/site-packages/scipy/linalg/_flapack.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x4906AA8: PyObject_Call (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BBE0C: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A08E5B: _PyEval_EvalCode (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A0925D: _PyFunction_Vectorcall (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BFA94: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48B9A82: function_code_fastcall (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48C040E: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==  Address 0x39eefb44 is 0 bytes after a block of size 4 alloc'd
==401176==    at 0x483F79B: malloc (in /nix/store/d1i8s1hzz72zdkcbzriqf18sgwp8fik6-valgrind-3.18.1/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==401176==    by 0x15E2D2E3: default_malloc (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x15E2D9FE: PyDataMem_UserNEW (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x15E8C6F5: PyArray_NewFromDescr_int (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x15E8C85F: PyArray_New (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x2746E2EB: array_from_pyobj (in /nix/store/gmcby9wc7c5agdilcqq8ngzvn0psz2gy-python3.9-scipy-1.8.0/lib/python3.9/site-packages/scipy/linalg/_flapack.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x273B44FD: f2py_rout__flapack_dsyevr (in /nix/store/gmcby9wc7c5agdilcqq8ngzvn0psz2gy-python3.9-scipy-1.8.0/lib/python3.9/site-packages/scipy/linalg/_flapack.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x4906AA8: PyObject_Call (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BBE0C: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A08E5B: _PyEval_EvalCode (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A0925D: _PyFunction_Vectorcall (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BFA94: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176== 
==401176== Invalid read of size 4
==401176==    at 0x174F5F24: dlarrv_ (in /nix/store/413dlxvbzrsfaakf00ffsgw7f1qapx3f-lapack-3/lib/liblapack.so.3)
==401176==    by 0x1756F766: dstemr_ (in /nix/store/413dlxvbzrsfaakf00ffsgw7f1qapx3f-lapack-3/lib/liblapack.so.3)
==401176==    by 0x1752F5DE: dsyevr_ (in /nix/store/413dlxvbzrsfaakf00ffsgw7f1qapx3f-lapack-3/lib/liblapack.so.3)
==401176==    by 0x273B45E0: f2py_rout__flapack_dsyevr (in /nix/store/gmcby9wc7c5agdilcqq8ngzvn0psz2gy-python3.9-scipy-1.8.0/lib/python3.9/site-packages/scipy/linalg/_flapack.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x4906AA8: PyObject_Call (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BBE0C: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A08E5B: _PyEval_EvalCode (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A0925D: _PyFunction_Vectorcall (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BFA94: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48B9A82: function_code_fastcall (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48C040E: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A08E5B: _PyEval_EvalCode (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==  Address 0x39eefb44 is 0 bytes after a block of size 4 alloc'd
==401176==    at 0x483F79B: malloc (in /nix/store/d1i8s1hzz72zdkcbzriqf18sgwp8fik6-valgrind-3.18.1/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==401176==    by 0x15E2D2E3: default_malloc (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x15E2D9FE: PyDataMem_UserNEW (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x15E8C6F5: PyArray_NewFromDescr_int (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x15E8C85F: PyArray_New (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x2746E2EB: array_from_pyobj (in /nix/store/gmcby9wc7c5agdilcqq8ngzvn0psz2gy-python3.9-scipy-1.8.0/lib/python3.9/site-packages/scipy/linalg/_flapack.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x273B44FD: f2py_rout__flapack_dsyevr (in /nix/store/gmcby9wc7c5agdilcqq8ngzvn0psz2gy-python3.9-scipy-1.8.0/lib/python3.9/site-packages/scipy/linalg/_flapack.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x4906AA8: PyObject_Call (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BBE0C: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A08E5B: _PyEval_EvalCode (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A0925D: _PyFunction_Vectorcall (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BFA94: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176== 
==401176== Invalid read of size 4
==401176==    at 0x174F71DA: dlarrv_ (in /nix/store/413dlxvbzrsfaakf00ffsgw7f1qapx3f-lapack-3/lib/liblapack.so.3)
==401176==    by 0x1756F766: dstemr_ (in /nix/store/413dlxvbzrsfaakf00ffsgw7f1qapx3f-lapack-3/lib/liblapack.so.3)
==401176==    by 0x1752F5DE: dsyevr_ (in /nix/store/413dlxvbzrsfaakf00ffsgw7f1qapx3f-lapack-3/lib/liblapack.so.3)
==401176==    by 0x273B45E0: f2py_rout__flapack_dsyevr (in /nix/store/gmcby9wc7c5agdilcqq8ngzvn0psz2gy-python3.9-scipy-1.8.0/lib/python3.9/site-packages/scipy/linalg/_flapack.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x4906AA8: PyObject_Call (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BBE0C: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A08E5B: _PyEval_EvalCode (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A0925D: _PyFunction_Vectorcall (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BFA94: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48B9A82: function_code_fastcall (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48C040E: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A08E5B: _PyEval_EvalCode (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==  Address 0x39eefb44 is 0 bytes after a block of size 4 alloc'd
==401176==    at 0x483F79B: malloc (in /nix/store/d1i8s1hzz72zdkcbzriqf18sgwp8fik6-valgrind-3.18.1/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==401176==    by 0x15E2D2E3: default_malloc (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x15E2D9FE: PyDataMem_UserNEW (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x15E8C6F5: PyArray_NewFromDescr_int (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x15E8C85F: PyArray_New (in /nix/store/fc68hcyr0cphb6mx7n9nngriill2azxw-python3.9-numpy-1.22.2/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x2746E2EB: array_from_pyobj (in /nix/store/gmcby9wc7c5agdilcqq8ngzvn0psz2gy-python3.9-scipy-1.8.0/lib/python3.9/site-packages/scipy/linalg/_flapack.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x273B44FD: f2py_rout__flapack_dsyevr (in /nix/store/gmcby9wc7c5agdilcqq8ngzvn0psz2gy-python3.9-scipy-1.8.0/lib/python3.9/site-packages/scipy/linalg/_flapack.cpython-39-x86_64-linux-gnu.so)
==401176==    by 0x4906AA8: PyObject_Call (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BBE0C: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A08E5B: _PyEval_EvalCode (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x4A0925D: _PyFunction_Vectorcall (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)
==401176==    by 0x48BFA94: _PyEval_EvalFrameDefault (in /nix/store/adnh88kc6cf2dc8fqfyvjmyyd86v52qn-python3-3.9.10/lib/libpython3.9.so.1.0)

...

Full python-annotated stack trace for the first of these: https://gist.github.com/risicle/b52b924f4e7353f1b306645499cb2e60

Assembly listing of write:

   0x00000000174ef00b <+1723>:  movsd  xmm8,QWORD PTR [rip+0x5f52b4]        # 0x17ae42c8
   0x00000000174ef014 <+1732>:  mov    ecx,DWORD PTR [rax]
   0x00000000174ef016 <+1734>:  mov    rax,QWORD PTR [rsp+0x120]
   0x00000000174ef01e <+1742>:  movsd  QWORD PTR [r11],xmm8
   0x00000000174ef023 <+1747>:  mov    DWORD PTR [rax],ecx
=> 0x00000000174ef025 <+1749>:  mov    DWORD PTR [rax+0x4],esi

Could it be trying to write a 64bit value into a 32bit allocation?

@risicle
Copy link
Contributor

risicle commented Mar 9, 2022 •
edited
Loading

According to debug symbols, this corresponds to the fortran line https://github.com/Reference-LAPACK/lapack/blob/v3.10.0/SRC/dlar1v.f#L402 no wait I've taken a wrong turn somewhere...

@risicle
Copy link
Contributor

risicle commented Mar 10, 2022 •
edited
Loading

Turns out it's more like https://github.com/xianyi/OpenBLAS/blob/0b678b19dc03f2a999d6e038814c4c50b9640a4e/lapack-netlib/SRC/dlar1v.f#L405 but pretty close the first time.

(backtrace with fortran debug info https://gist.github.com/risicle/c20d85a6922a9cbc92312aa1b0c1f65f)

mweinelt added a commit that referenced this pull request Mar 12, 2022
@mweinelt
Revert "python3Packages.numpy: 1.21.5 -> 1.22.2"
cc7f038 
This reverts commit 940af4b.

Causes problems with scikit-learn, details can be found over in
#156578.
mweinelt added a commit that referenced this pull request Mar 13, 2022
@mweinelt
Revert "python3Packages.numpy: 1.21.5 -> 1.22.2"
9b74eef 
This reverts commit 940af4b.

Causes problems with scikit-learn, details can be found over in
#156578.
@mweinelt mweinelt changed the title python3Packages.numpy: 1.21.5 -> 1.22.1 python3Packages.numpy: 1.21.5 -> 1.23.1 Jul 14, 2022
@mweinelt
Copy link
Member Author

mweinelt commented Jul 14, 2022

@ofborg build python3Packages.scikit-learn

@ofborg ofborg bot added 10.rebuild-darwin: 2501-5000 This PR causes many rebuilds on Darwin and should target the staging branches. and removed 10.rebuild-darwin: 1001-2500 This PR causes many rebuilds on Darwin and should most likely target the staging branches. labels Jul 14, 2022
@mweinelt mweinelt closed this Jul 15, 2022
@mweinelt mweinelt deleted the numpy branch July 15, 2022 17:56
@Scriptkiddi
Copy link
Contributor

Scriptkiddi commented Jul 16, 2022

any reason for this being closed? like a new pr?

@risicle
Copy link
Contributor

risicle commented Jul 16, 2022

#181620

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FRidh FRidh Awaiting requested review from FRidh

@jonringer jonringer Awaiting requested review from jonringer

Assignees

No one assigned

Labels

6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 2501-5000 This PR causes many rebuilds on Darwin and should target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 2501-5000 This PR causes many rebuilds on Linux and should target the staging branches.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mweinelt @FRidh @Scriptkiddi @risicle