Move 32bit CI to weekly by WilliamJamieson · Pull Request #15018 · astropy/astropy

WilliamJamieson · 2023-07-03T15:59:00Z

Description

I believe arm7 is the arch of some of the 32bit native raspberry pi computers. So it should be a good test for continued 32 bit support. This pull request adds it and a test for 32bit support to the weekly cron.

closes #15007

github-actions · 2023-07-03T15:59:38Z

github-actions · 2023-07-03T15:59:43Z

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

pllim · 2023-07-03T18:43:38Z

We also have to put the double test somewhere. It is currently piggybacking on the old 32-bit job. And with this PR, we'll also have to remove the old 32-bit job. And since the old job is a required job, I'll have to update branch protection rule after merge.

WilliamJamieson · 2023-07-03T20:36:38Z

+            uname -a
+            getconf LONG_BIT


These only exist to so we can confirm that some of the jobs are 32 bit.

Your comment above is a little incomplete, so I am trying to understand more. Do you mean that getconf LONG_BIT would just crash if it is not a true 32-bit machine? Or is it suppose to print something out to the log?

This prints out how many bits a floating point is by default in the architecture. It should return 32, which it does.

2023-07-07T16:50:23.8975207Z run: uname -a getconf LONG_BIT python3 -m venv --system-site-packages tests source tests/bin/activate ASTROPY_USE_SYSTEM_ALL=1 pip3 install -e .[test] python3 -m pytest

I don't see any number printed out here.

uname -a getconf LONG_BIT

Outputs are right here:

https://github.com/astropy/astropy/actions/runs/5488698114/jobs/10001903172?pr=15018#step:3:1000

https://github.com/astropy/astropy/actions/runs/5488698114/jobs/10001903172?pr=15018#step:3:1001

Anyway, we can make it print out LONG_BIT=32 instead of just 32? It is really hard to find.

WilliamJamieson · 2023-07-03T20:37:59Z

+    !cov: pytest --pyargs astropy {toxinidir}/docs {env:MPLFLAGS} {posargs}
+    cov: pytest --pyargs astropy {toxinidir}/docs {env:MPLFLAGS} --cov astropy --cov-config={toxinidir}/pyproject.toml --cov-report xml:{toxinidir}/coverage.xml {posargs}

-    double: python -c 'import sys; from astropy import test; test(); sys.exit(test())'
+    double: pytest --pyargs astropy {toxinidir}/docs {env:MPLFLAGS} {posargs}


@saimn and @astrofrog will this work for the double tests? It just runs all of the tests twice in the same tox environment.

astrofrog · 2023-07-04T08:23:40Z

The double test should not also be the parallel test, because the parallel test actually isolates tests into subprocesses whereas the double test is checking for changes in the global state which won't happen if the tests run in subprocesses.

larrybradley · 2023-07-10T15:07:30Z

It appears that there are no numpy 1.22 packages for these architectures for ubuntu-latest (22.04). They only go up to numpy 1.21.5 (https://packages.ubuntu.com/search?keywords=python-numpy). And I don't see arm7 as a supported ubuntu architecture.

A similar workflow failed for me for photutils (numpy 1.22+). It was trying to build numpy from source (e.g., https://github.com/astropy/photutils/actions/runs/5467812504/jobs/9954604026).

WilliamJamieson · 2023-07-10T15:33:09Z

It appears that there are no numpy 1.22 packages for these architectures for ubuntu-latest (22.04). They only go up to numpy 1.21.5 (packages.ubuntu.com/search?keywords=python-numpy). And I don't see arm7 as a supported ubuntu architecture.

A similar workflow failed for me for photutils (numpy 1.22+). It was trying to build numpy from source (e.g., astropy/photutils/actions/runs/5467812504/jobs/9954604026).

I don't understand what you are getting at here, it ran perfectly fine see:

pllim · 2023-07-10T15:48:20Z

I think what @larrybradley was trying to say is that we cannot test any of those exotic archs no more if we drop numpy 1.21 now, unless there is a way to ask them to provide numpy 1.22 or later.

larrybradley · 2023-07-10T15:53:59Z

@WilliamJamieson Those jobs are using numpy 1.21.5 (e.g., https://github.com/astropy/astropy/actions/runs/5488698114/jobs/10001902806#step:3:813). This PR must have been created before requiring numpy >= 1.22.

WilliamJamieson · 2023-07-10T17:26:09Z

@WilliamJamieson Those jobs are using numpy 1.21.5 (e.g., astropy/astropy/actions/runs/5488698114/jobs/10001902806#step:3:813). This PR must have been created before requiring numpy >= 1.22.

See my update, 8558a28, simply switching to the rolling ubuntu release solves this issue:

https://github.com/astropy/astropy/actions/runs/5510400099/jobs/10044557358#step:3:1156

pllim · 2023-07-10T18:14:44Z

        with:
          arch: ${{ matrix.arch }}
-          distro: ubuntu_latest
+          distro: ubuntu_rolling


Is rolling too unstable? Will we see breakage that we cannot do anything about?

Its the only one available by our workflow which has numpy 1.22+ pre-built.

So theoretically, we can bump it down to latest again when the distro people catch up?

Not like we have a choice here... but we might have to rethink if this proves to be too unstable. Unless @olebole has a better idea?

We can use the base_image option and pick a docker image from docker hub that works. I think the debian bookworm image should work.

Note rolling is not the unstable devel release. rolling is the latest release of Ubuntu 23.04 lunar. Its the rolling pulled from Dockerhub: https://hub.docker.com/_/ubuntu

astrofrog

Looks good, thanks!

arm7 is the arch of some of the 32bit native raspberry pi computers. So it should be a good test for continued 32 bit support

WilliamJamieson · 2023-07-13T21:56:27Z


          run: |
+            uname -a
+            echo "LONG_BIT="$(getconf LONG_BIT)


@pllim This should return

LONG_BIT=32

or

LONG_BIT=64

Does that seem easier to pick out in the log?

pllim

Thanks!

pllim · 2023-07-14T05:01:08Z

I updated the branch protection rules.

WilliamJamieson mentioned this pull request Jul 3, 2023

Should astropy move 32bit CI to weekly. #15007

Closed

WilliamJamieson added the Extra CI Run cron CI as part of PR label Jul 3, 2023

github-actions Bot added the testing label Jul 3, 2023

WilliamJamieson force-pushed the feature/move_32_bit branch 2 times, most recently from 262660f to d5e1124 Compare July 3, 2023 19:37

WilliamJamieson added the no-changelog-entry-needed label Jul 3, 2023

WilliamJamieson requested review from astrofrog, pllim and saimn July 3, 2023 20:35

WilliamJamieson marked this pull request as ready for review July 3, 2023 20:35

WilliamJamieson commented Jul 3, 2023

View reviewed changes

WilliamJamieson commented Jul 4, 2023

View reviewed changes

Comment thread .github/workflows/ci_workflows.yml Resolved

WilliamJamieson changed the title ~~TST: Move 32bit CI to weekly~~ Move 32bit CI to weekly Jul 5, 2023

WilliamJamieson mentioned this pull request Jul 5, 2023

Drop support for Numpy 1.21 #15006

Merged

WilliamJamieson force-pushed the feature/move_32_bit branch 2 times, most recently from e025f71 to 4268234 Compare July 7, 2023 16:49

WilliamJamieson force-pushed the feature/move_32_bit branch 2 times, most recently from 47996a8 to 8558a28 Compare July 10, 2023 16:11

This was referenced Jul 10, 2023

Fix weekly CI Architecture builds astropy/photutils#1575

Merged

Expand and fix the s390x workflow asdf-format/asdf#1580

Closed

pllim reviewed Jul 10, 2023

View reviewed changes

WilliamJamieson force-pushed the feature/move_32_bit branch 3 times, most recently from 984a2e2 to 1b1925b Compare July 13, 2023 18:45

astrofrog approved these changes Jul 13, 2023

View reviewed changes

WilliamJamieson added 4 commits July 13, 2023 17:53

Add arm7 weekly cron

3f5358e

arm7 is the arch of some of the 32bit native raspberry pi computers. So it should be a good test for continued 32 bit support

Remove 32-bit from standard PR workflow.

f8da4a1

Try ubuntu rolling

9369c2d

Make system LONG_BIT easier to pick out

5f55eed

WilliamJamieson force-pushed the feature/move_32_bit branch from 1b1925b to 5f55eed Compare July 13, 2023 21:54

WilliamJamieson commented Jul 13, 2023

View reviewed changes

WilliamJamieson requested a review from pllim July 13, 2023 22:21

pllim approved these changes Jul 14, 2023

View reviewed changes

pllim merged commit 98b399d into astropy:main Jul 14, 2023

pllim added this to the v6.0 milestone Jul 14, 2023

WilliamJamieson deleted the feature/move_32_bit branch July 14, 2023 05:26

pllim mentioned this pull request Sep 18, 2023

Backport PR #15339 on branch v5.3.x (Attempt to fix wheel building), removed parallel-and-32bit job #15340

Merged

WilliamJamieson mentioned this pull request Sep 19, 2023

Cannot build astropy in CI for exotic archs #15351

Closed

Conversation

WilliamJamieson commented Jul 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

github-actions Bot commented Jul 3, 2023

Uh oh!

github-actions Bot commented Jul 3, 2023

Uh oh!

pllim commented Jul 3, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

WilliamJamieson Jul 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

astrofrog commented Jul 4, 2023

Uh oh!

Uh oh!

larrybradley commented Jul 10, 2023

Uh oh!

WilliamJamieson commented Jul 10, 2023

Uh oh!

pllim commented Jul 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

larrybradley commented Jul 10, 2023

Uh oh!

WilliamJamieson commented Jul 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

WilliamJamieson Jul 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

astrofrog left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pllim left a comment

Choose a reason for hiding this comment

Uh oh!

pllim commented Jul 14, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

WilliamJamieson commented Jul 3, 2023 •

edited

Loading

WilliamJamieson Jul 3, 2023 •

edited

Loading

pllim commented Jul 10, 2023 •

edited

Loading

WilliamJamieson commented Jul 10, 2023 •

edited

Loading

WilliamJamieson Jul 10, 2023 •

edited

Loading