TST: GitHub Actions as CI by pllim · Pull Request #10388 · astropy/astropy

pllim · 2020-05-22T21:53:02Z

Update: Due to Travis CI changes in pricing, we decided to push ahead with this as soon as we have a minimal viable product. We will fix the rest of the jobs in follow-up PRs.

~~This is a small proof-of-concept of using GitHub Actions as CI. Might expand if there is buy-in, or abandon if there isn't.~~

Pros

Artifact support but I haven't tried to use it yet.
Better integration with GitHub UI.
- The cancel button is easier to access than Azure Pipelines.
Unlike Azure, no need for special Microsoft account or let Microsoft access GitHub credentials. (The latter is probably moot point, har har...).
Performance on par with Azure (theoretically, considering they probably share the same servers or something...).
Potentially useful for other things too, like bot work and delivery.

Cons

No anchors or templating.
No allowed failures. Also see Please support something like "allow-failure" for a given job actions/runner#2347
No built in support for skip ci.
- On the bright side, I can use [skip ci] to skip all the other CIs and only run Actions. (Har har...)
No ccache? If there is one, it is not documented.
Cannot re-run only one job.
No extra architecture support (e.g., ARM64) without hosting self-runners.
No re-run with SSH like CircleCI.

CI jobs and features to implement (please check off if done)

From .travis.yml:

From .circleci/config.yml:

32-bit and parallel (custom astropy.cfg, Python 3.7 -n=4 --durations=50, Python 3.8 py38-test-double)
image-tests-mpl302
image-tests-mpl310
image-tests-mpldev

From azure-piplines.yml:

Build wheels and publish to PyPI for release. (@astrofrog and @Cadair decided to keep Azure for now.)

TODO

Possible links of interest

Thanks to everyone who provided these resources:

Cadair · 2020-05-24T16:58:42Z

The cancel button is easier to access than Azure Pipelines.

Don't they both ingrate with the GH actions UI for this? You can restart pipelines from the actions tab as well.

Cadair · 2020-05-26T17:14:21Z

The GH account integration with pipelines has improved a lot in the last few months.

astrofrog · 2020-11-02T18:08:04Z

+  # Full tests with coverage checks.
+  coverage_test:
+    runs-on: ubuntu-latest
+    needs: [code_style, initial_tests]


I think we could try and keep things simple and not worry about replicating stages for now - we could have a single jobs section with an include matrix?

Instead of stages I think we could rely on Actions' fast-fail functionality (default) which means that all jobs stop if one fail. In practice this means if e.g. code style fails it won't waste time on longer builds for example.

Up to you. I was just trying to recreate what we had at the time.

I think the issue is that the stages makes everything much more verbose. I think I would prefer if we could make things more concise and avoid repetition and then decide if adding stages back are worth it?

astrofrog · 2020-11-03T11:04:38Z

+# The following is needed to avoid issues if e.g. Matplotlib tries
+# to open a GUI window.
+env:
+  SETUP_XVFB: True


I think this was a ci-helpers thing

Yeah, this PR been open for a while and probably need upgrades.

astrofrog · 2020-11-12T11:20:49Z

@pllim - since there was a conflict, and to pick up the fact actions now works in PRs, I took the liberty of rebasing this and removing the temporary commit. I also added a commit which simplifies the configuration a lot by getting rid of the stages (this reduces a lot of configuration duplication). Once we are happy with the config, we could change fail-fast to true which means that if some of the faster builds (e.g. code style or minimal deps) fail the whole CI stops, which in effect reproduces what we were trying to accomplish with stages. We could add stages back as you had them of course, but I think we should aim to have a CI config file as simple as possible. Finally, I've removed jobs from the Travis configuration which are now in Actions. I have a backup of your branch if needed for any reason though. I haven't yet added the ability to use clang for one of the Linux jobs.

astrofrog · 2020-11-12T11:23:29Z

One thing we should decide is how to handle allowed failures - since each job appears as one entry here maybe we don't actually need to replicate this functionality, and that way remote data and numpy dev failures won't be hidden away but are also easy to ignore.

astrofrog · 2020-11-12T12:46:17Z

I've also now copied over most cron jobs, except the arm and s390x arch ones which I can do a follow up PR for later. I made a separate file for cron otherwise it gets a bit messy to skip some jobs and not others for cron and I haven't found a way to hide them in pull requests then. I've also added a TMP commit to enable cron builds here so we can check they run.

astrofrog · 2020-11-12T12:50:51Z

The main thing I haven't implemented is the use of clang which was specified in the Travis file for some jobs. Is that actually something we want/need? We could get rid of that and just make the MacOS X build be non-cron which would then by default use clang?

@Cadair - you could always push your changes here to move the image tests away from CircleCI?

Cadair · 2020-11-12T12:57:43Z

As far as i can tell actions artifacts aren't up to it. I suggest we just leave circle for now and focus on killing off Travis.

astrofrog · 2020-11-12T13:28:04Z

Ok so looking at the failures for now:

~~test_update_parallel was skipped on Travis but also fails on Actions so I'm now pushing a commit to skip it on all CI~~ (fixed)
Python 3.7 with oldest supported version of all dependencies fails because there are some open files after the tests (mainly FITS files I think). @saimn any idea what could be going on there?
The link checker is failing because https://docs.astropy.org/en/v4.0/whatsnew/4.0.html is broken (any ideas why?)
There are some real failures with Numpy dev I think (cc @mhvk)
Python dev (now 3.10) fails because Numpy isn't compatible yet. Maybe we can just comment it out for now so we can keep an eye on what the latest failures are

saimn · 2020-11-12T14:02:12Z

Re-clang: yes I think we had clang on linux because the macos queue on Travis was too slow. So if we can have a macos job by default, we don't need clang on linux.

astrofrog · 2020-11-12T14:04:25Z

@saimn - ok let's do that. The MacOS X builds start up pretty quickly on Actions.

pllim · 2020-11-12T18:35:34Z

@@ -1,4 +1,4 @@
-name: Run tests
+name: CI


Note: This name has to match what is on README

[ci skip]

bsipocz

minor comments only, no need to address them here, they are just thought for the follow-up.

bsipocz · 2020-11-12T19:17:06Z

+on:
+  schedule:
+    # run every Monday at 6am UTC
+    - cron: '0 6 * * 1'


is this frequent enough? We used to run cron jobs daily.

Good point. We can follow up as separate PR.

bsipocz · 2020-11-12T19:18:49Z

 =======

-|Travis Status| |CircleCI Status| |Coverage Status| |PyPI Status| |Documentation Status|
+|Actions Status| |CircleCI Status| |Coverage Status| |PyPI Status| |Documentation Status|


travis isn't completely removed yet, so maybe keep both?

It doesn't run no more. We can add it back if we decide to pay.

ok, indeed. I only said it based on that the config remained.

Cadair

astrofrog · 2020-11-12T19:24:16Z

🎉

mwcraig · 2020-11-12T21:24:11Z

Nice work! 🎉

TST: GitHub Actions as CI

bsipocz · 2020-11-23T07:53:51Z

I've added this to the backport as a first step of getting #11050 addressed, and to get some CI on the 4.2.x branch. I'm certain that more backports and/or fine-tuning will be necessary (cleaning up cron as that is not run on non-master branches, etc., adding back 3.6 testing, etc.), but still getting this is superb for now.

pllim added Work in progress Experimental testing labels May 22, 2020

pllim added this to the v4.2 milestone May 22, 2020

pllim force-pushed the last-actions-hero branch from 025d81d to f16755d Compare May 22, 2020 21:59

This comment has been minimized.

Sign in to view

pllim mentioned this pull request Aug 14, 2020

TST: Use GitHub Actions ejeschke/ginga#872

Merged

2 tasks done

pllim removed the Work in progress label Sep 16, 2020

This was referenced Oct 13, 2020

TST: Move Windows CI job to Actions #10844

Closed

TST: Allow Windows CI to fail for now #10846

Merged

pllim modified the milestones: v4.2, v5.0 Oct 21, 2020

astrofrog reviewed Nov 2, 2020

View reviewed changes

astrofrog reviewed Nov 3, 2020

View reviewed changes

astrofrog force-pushed the last-actions-hero branch from f16755d to d0c1f0f Compare November 12, 2020 11:12

astrofrog force-pushed the last-actions-hero branch from c26031d to 3255b7b Compare November 12, 2020 12:32

Cadair reviewed Nov 12, 2020

View reviewed changes

Comment thread .github/workflows/ci_workflows.yml Resolved

astrofrog force-pushed the last-actions-hero branch 2 times, most recently from 913de6f to 41afca2 Compare November 12, 2020 12:38

astrofrog force-pushed the last-actions-hero branch from 542d24b to 1f0043b Compare November 12, 2020 13:42

Fix coverage reporting

34703b5

pllim force-pushed the last-actions-hero branch from 966df97 to f0d7cd0 Compare November 12, 2020 17:35

TST: Remove --open-files from oldest-deps job

a7e7500

pllim force-pushed the last-actions-hero branch from f0d7cd0 to a7e7500 Compare November 12, 2020 18:03

pllim commented Nov 12, 2020

View reviewed changes

Comment thread .github/workflows/ci_workflows.yml Resolved

pllim marked this pull request as ready for review November 12, 2020 18:23

pllim commented Nov 12, 2020

View reviewed changes

TST: Fix badge

dc0c21b

[ci skip]

pllim force-pushed the last-actions-hero branch from b5e87d3 to dc0c21b Compare November 12, 2020 18:54

bsipocz approved these changes Nov 12, 2020

View reviewed changes

pllim merged commit 7a97f04 into astropy:master Nov 12, 2020

pllim deleted the last-actions-hero branch November 12, 2020 19:21

pllim mentioned this pull request Nov 12, 2020

WIP: Azure pipelines configuration #8445

Closed

4 tasks

Cadair approved these changes Nov 12, 2020

View reviewed changes

bsipocz modified the milestones: v4.3, v4.2 Nov 23, 2020

bsipocz pushed a commit that referenced this pull request Nov 23, 2020

Merge pull request #10388 from pllim/last-actions-hero

652ef59

TST: GitHub Actions as CI

bmorris3 mentioned this pull request Jan 27, 2021

Travis-CI -> Github Actions exoclime/THOR#48

Closed

Conversation

pllim commented May 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Cadair commented May 24, 2020

Uh oh!

This comment has been minimized.

Cadair commented May 26, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

astrofrog commented Nov 12, 2020

Uh oh!

astrofrog commented Nov 12, 2020

Uh oh!

Uh oh!

astrofrog commented Nov 12, 2020

Uh oh!

astrofrog commented Nov 12, 2020

Uh oh!

Cadair commented Nov 12, 2020

Uh oh!

astrofrog commented Nov 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

saimn commented Nov 12, 2020

Uh oh!

astrofrog commented Nov 12, 2020

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bsipocz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Cadair left a comment

Choose a reason for hiding this comment

Uh oh!

astrofrog commented Nov 12, 2020

Uh oh!

mwcraig commented Nov 12, 2020

Uh oh!

bsipocz commented Nov 23, 2020

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

pllim commented May 22, 2020 •

edited

Loading

astrofrog commented Nov 12, 2020 •

edited

Loading