Skip to content

CI: Windows on ARM experiment: numpy versions (not for merging) - #1005

Closed
bashtage wants to merge 3 commits into
mainfrom
a90-winarm-versions
Closed

bashtage wants to merge 3 commits into
mainfrom
a90-winarm-versions

Conversation

@bashtage

@bashtage bashtage commented Nov 4, 2026 •

Copy link
Copy Markdown
Owner

Experimental, not for merging. One of four experiments on the access violations in numpy.linalg on Windows on ARM that #1003 found. With OPENBLAS_NUM_THREADS=1, inv and solve of small positive definite matrices crash in numpy 2.5.4's OpenBLAS, and numpy alone reproduces it. Each experiment has its own pull request, so that they run at the same time and each can change without restarting the others: #1003 (arch's tests), #1004 (the calls and threads), #1005 (the numpy versions), #1006 (the OpenBLAS routines) and #1007 (SciPy, the stack and faulthandler).

Like #1003, this removes the Azure pipeline (azure-pipelines.yml) and the wheel builds (cron-test.yml), so that no tests run here. arch is not built: the job installs only numpy (and SciPy where needed) from wheels.

What this one runs

inv, solve, det, cholesky and matmul with 1 OpenBLAS thread for numpy 2.3.0 (the first with win_arm64 wheels), 2.3.5, 2.4.1, 2.4.6, 2.5.0 and 2.5.4, and the nightly build, all on Python 3.13, and 2.5.4 and the nightly on Python 3.14. This shows whether the crash is new, and whether it's fixed in numpy's main branch. A numpy maintainer asked on numpy#29442 whether it was fixed. Each report has the version of OpenBLAS that the numpy wheel bundles.

How it works

ci/github-actions/winarm_blas.py, the same in all four pull requests, makes every call in a child process. A child walks through the sizes (1 to 64, then every 8th to 160) and prints start CALL N before each call. When it crashes or hangs (60 s), the size is recorded and a new child goes on with the next size. Each size that crashed is then tried alone, in a new process. So a map takes minutes rather than a process for every size, which is what made the size map in #1003 so slow. The matrices are built without BLAS (einsum), so the call under test is the only BLAS call.

The job passes when the map is complete, whether or not anything crashed. The results are in the job summary, in the log as they come, and in the report.md artifact. Each report starts with the builds and what OpenBLAS reports at run time (kernel, threads, configuration) and the CPU.

Checked locally (x64)

  • actionlint and check-jsonschema (GitHub workflow schema) report no problems. The script is clean under ruff, black, isort, flake8, bandit and pylint.
  • Every command of the four workflows runs to the end on x64, in under a minute in total, with no crashes. selftest crashes at size 5 and is found, the walk goes on, and the crash is found again alone.
  • The ctypes calls into numpy's OpenBLAS (ILP64 on x64) and SciPy's (LP64) check their results without BLAS.

🤖 Generated with Claude Code

The experiments on the numpy.linalg crashes on Windows on ARM (see #1003)
run in their own pull requests. Remove the Azure pipeline and the wheel
builds, the workflows that run the tests, so that these pull requests do
not use the CI queues. Not for merging.
Maps the sizes at which numpy.linalg calls, SciPy's LAPACK and BLAS, and
the LAPACK and BLAS routines of numpy's OpenBLAS (called directly with
ctypes) crash. A child process walks through the sizes, and after a crash
or a hang a new child continues with the next size, so a map takes
minutes rather than a process for every size. Each size that crashed is
then tried alone. Options set the OpenBLAS threads in the environment or
at run time, the stack of the thread that makes the calls, and
faulthandler. Not for merging.
Maps the sizes at which inv, solve, det, cholesky and matmul crash with 1
OpenBLAS thread for numpy 2.3.0 to 2.5.4, the first and last of each minor
release with win_arm64 wheels, and the nightly build. Not for merging.

result["scipy"] = scipy.__version__
result["scipy openblas"] = runtime("scipy")
except ImportError:
@bashtage bashtage closed this Nov 5, 2026
@bashtage
bashtage deleted the a90-winarm-versions branch November 11, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants