CI: run some wheel build jobs by default, and clean up the rest #29540

rgommers · 2025-08-11T19:57:43Z

Now that we have https://github.com/numpy/numpy-release, we can remove most wheel build jobs from this repo and at the same time improve the default set of jobs running in CI by adding some wheel build jobs to the mix (none got run by default before this PR).

Up for discussion:

Exactly what do we run here? My first stab at it was 1 wheel per platform, minus the ones where we don't vendor OpenBLAS.
Do we run the full or fast test suite?
Are there other jobs we can clean up now because of overlap (like the linux_musl.yml one removed here)
Still TBD: how exactly do we deal with some remaining overlap between files in the numpy and numpy-release repos? xref Sync the code here with upstream numpy/numpy numpy-release#6 which @mattip just opened

andyfaff · 2025-08-12T00:34:25Z

The linux_musl.yml file was supposed to be the place for examining musl, it just so happened to be built/installed through a wheel. I still think it makes sense to keep a musl linux run in this file, but we probably don't need to send it through a wheel first. We may find out at a later stage that we want to amend some options for such a test run.

What pain points do the full/fast runs tend to highlight? We should probably run full on numpy-releases (that gets triggered less), so we can probably run fast in the wheels here, and save some full runs for builds that don't go through wheel building.

rgommers · 2025-08-12T08:53:45Z

What pain points do the full/fast runs tend to highlight?

I just had a look at what's decorated with mark.slow, and it's not that much - mostly f2py tests, I/O tests, some other compile tests, and static typing tests (which won't run by default anyway). So the extra value of running them on more then a handful of platforms is pretty low.

I still think it makes sense to keep a musl linux run in this file,

I really don't think it adds any value. It made a lot of sense when we did not run wheel build jobs on PRs, but when we run a musllinux x86-64 and aarch64 wheel build job already, the extra job is 100% duplicate.

Other jobs that can go, for the same reason, are:

the Windows x86-64 and arm64 ones that build with scipy-openblas64 in windows.yml
the plain Linux one on ubuntu-22.04-arm in linux.yml

If we remove all those, that still leaves us with four extra jobs after this PR, which seems acceptable for the win of testing wheel builds by default.

Still TBD: how exactly do we deal with some remaining overlap between files in the numpy and numpy-release repos?

Still thinking about this. Less duplication is good for maintainability. We need to have at least some files in this repo so we can do wheel builds. We don't need the licensing related stuff here, because we're not distributing wheels from this repo anymore, and there's a benefit to removing it (e.g., no license checkers will notice LGPL etc. in this repo and raise flags by default).

The main things that leaves are:

cibw_before_build.sh, cibw_test_command.sh and repair_windows.sh
the pyproject.toml cibuildwheel config
requirements files

I'm inclined to say for (1) that we avoid duplication and use the files in this repo from numpy-release. For (2) the configs are slightly different (e.g., pyodide is added), but could be reused. For (3) it's best kept separate with a check on no mismatches probably, and ensuring the numpy-release ones are minimal; the ones in this repo are very messy with many unnecessary build and test dependencies added.

This accounts for the actual release builds moving to the separate `numpy-release` repo. We want to keep a decent subset of wheel builds in the main repo and always run them, just like other CI jobs that test some config that matters.

These configs are covered by `wheels.yml`, which is now part of the regular test suite rather than off-by-default on PRs.

rgommers · 2025-09-09T19:41:18Z

This is ready now. I went with the full test suite for now, since the extra time is only a couple of minutes per job and I'd like to avoid testing the slow tests on niche architectures in numpy-release only for now.

andyfaff · 2025-09-10T01:27:19Z

tools/wheels/cibw_before_build.sh

+    # at it in `repair_windows.sh` (needed only on Windows because of the lack
+    # of RPATH support).
+    if [[ $RUNNER_OS == "Windows" ]]; then
+        python <<EOF


This bit is a little complicated to follow for me because the indentation is changing.

Yeah, this took me a while. The old version was super confusing, it was unclear when reading the code that PKG_CONFIG_PATH was not an environment variable.

The diff is hard to read; just looking at the old and new versions side by side should be helpful. What it does isn't changing IIRC, the new version should just be easier to understand.

+1 from me, as the previous version's author

andyfaff · 2025-09-10T01:33:19Z

tools/wheels/LICENSE_linux.txt

When the licences are removed we're losing a record of the code that's being bundled (OpenBLAS, etc).
pyproject.toml says

The licenses for these vendored components are dynamically included in the build process for PyPI wheels.

I can see these files are now in numpy-release, but isn't it important to have a record of them here somewhere. Are there going to be improvements in how we bundle these extra licences in the wheel build process?

I think it's actually somewhat important to not have GPL licenses in this repo, because there is nothing inside this repo that's GPL, LGPL or otherwise copyleft.

I'd really like to conceptually separate NumPy the project and how its sources are licensed - which matters to anyone building/redistributing numpy - and wheels on PyPI, which are only one way of distributing numpy, and a quite unusual way at that. If you get numpy from any other source than PyPI, it will not contain copyleft components.

Are there going to be improvements in how we bundle these extra licences in the wheel build process?

Hopefully yes. See numpy/numpy-release#11

andyfaff · 2025-09-10T01:34:42Z

pyproject.toml

 PKG_CONFIG_PATH="/project/.openblas"
-LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/project/.openblas/lib"
-
-[tool.cibuildwheel.macos]


So these entries are being removed because we only need them in numpy-release now?

No we still have macOS wheel builds here in wheels.yml. I actually improved how this bundling works, so messing with DYLD_LIBRARY_PATH is no longer necessary. If you look at https://github.com/numpy/numpy-release/blob/main/cibuildwheel.toml, you won't find a macos entry there either.

.github/workflows/linux_musl.yml

.github/workflows/linux.yml

andyfaff · 2025-09-10T01:37:54Z

.github/workflows/wheels.yml

          - [ubuntu-22.04-arm, manylinux_aarch64, ""]
          - [ubuntu-22.04-arm, musllinux_aarch64, ""]
          - [macos-13, macosx_x86_64, openblas]
-          - [macos-13, macosx_x86_64, accelerate]


Did you want to remove all macos-13 runs here?

Should we move them to azure where there are still native macos x86_64 runners?

Can we do that separately based on what comes out of the discussion at gh-29728?

I haven't yet looked into what Azure plans to do with their platform support. Also, I hope we'll get rid of Azure completely ....

For this one, my thinking was that there's another job that tests Accelerate LP64 and ILP64 at once (see here, plus this particular job is quite slow, so I wanted to remove this one as duplicate.

.github/workflows/windows.yml

andyfaff · 2025-09-10T10:30:14Z

I'm happy for this to be merged. I'll do so tomorrow morning if it's not done by then.

mattip · 2025-09-10T11:08:02Z

Thanks @rgommers

rgommers · 2025-09-10T11:09:35Z

Thanks for the reviews! Let me look at the last open comment about the python64bit_openblas_winarm64 job as a potential follow-up.

rgommers added the component: CI label Aug 11, 2025

rgommers added this to the 2.4.0 release milestone Sep 8, 2025

This was referenced Sep 9, 2025

NumPy 2.3.3 wheels do not upload to staging. #29705

Closed

REL: review the numpy-release repo and switch over to that for releases #29462

Closed

rgommers added 2 commits September 9, 2025 15:49

CI: remove redundant Windows, musllinux jobs

6a16e90

These configs are covered by `wheels.yml`, which is now part of the regular test suite rather than off-by-default on PRs.

rgommers force-pushed the ci-wheels-cleanup branch from 1495100 to 7a4901f Compare September 9, 2025 14:18

CI: sync wheel build cleanups back from numpy-release repo

7888892

rgommers force-pushed the ci-wheels-cleanup branch from 7a4901f to 7888892 Compare September 9, 2025 18:56

rgommers marked this pull request as ready for review September 9, 2025 19:40

andyfaff reviewed Sep 10, 2025

View reviewed changes

.github/workflows/windows.yml Show resolved Hide resolved

mattip merged commit 295a14a into numpy:main Sep 10, 2025
76 checks passed

rgommers deleted the ci-wheels-cleanup branch September 10, 2025 11:08

rgommers mentioned this pull request Sep 19, 2025

Sync the code here with upstream numpy/numpy numpy/numpy-release#6

Closed

Uh oh!

CI: run some wheel build jobs by default, and clean up the rest #29540

CI: run some wheel build jobs by default, and clean up the rest #29540

Uh oh!

Conversation

rgommers commented Aug 11, 2025

Uh oh!

andyfaff commented Aug 12, 2025

Uh oh!

rgommers commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rgommers commented Sep 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andyfaff commented Sep 10, 2025

Uh oh!

Uh oh!

mattip commented Sep 10, 2025

Uh oh!

rgommers commented Sep 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rgommers commented Aug 12, 2025 •

edited

Loading